KohonAnts: A Self-Organizing Ant Algorithm for Clustering and Pattern Classification

KohonAnts: A Self-Organizing Ant Algorithm for Clustering and Pattern Classification Antonio M. Mora , C.M. Fernandes, J.J. Merelo, J.L.J. Laredo Dpto. ATC. UNIVERSITY OF GRANADA (Spain) V. Ramos, A.C. Rosa LASEEB-ISR/IST. UNIVERSITY OF LISBON (Portugal) ALIFE XI

INDEX Ant Algorithms Kohonen’s Self Organizing Maps KohonAnts Experiments and Results Conclusions and Future Work ALIFE XI

Ant Algorithms Natural Ants ALIFE XI Ants live in colonies and cooperate searching for the benefit of the colony instead of their own. Ants move following (smelling) and depositing a chemical substance known as pheromone (which is also evaporated). This ‘comunication’ method is known as stigmergy . They can find the shortest path between their nest and the food location. They build clusters with their larvae and also with their dead bodies.

Ant Algorithms Main Features ALIFE XI There are many agents called (artificial) ants . All of them move in a graph following and depositing (artificial) pheromone . They cooperate to find a solution (usually every ant yields a complete solution). There are some formulae applied in the running: state transition rule  decides the next step for each ant pheromone updating  contribution and evaporation evaluation function  assigns the cost to every solution

Ant Algorithms Ant System Model ALIFE XI Introduced by Chialvo and Millonas in 1995. Ants move in a grid . Each one is represented by its position r , and orientation to move  . They decide where to move without consider their previous movements. They apply the so-called response function , which depends on 3 parameters:   pheromone density of the cell   degree of randomness with which an ant follow the gradient of pheromone   factor of ant’s ability to sense pheromone

Kohonen’s SOM It is an unsupervised Artificial Neural Network. There is only an input and an output layer. There is a Competitive learning (only one neuron is activated for an input pattern). It Maps high-dimensional data into a two-dimensional representation space (usually rectangular or hexagonal grid). After training phase, neurons with similar weights tend to cluster on the map . Main Features ALIFE XI

Kohonen’s SOM N neurons in the input layer, M neurons in the output layer. Each input neuron is connected with all the output ones. Every input sample is associated to an input neuron. Each neuron of the output layer has associated a weight vector Wj , as the same dimension as the input data. General Structure ALIFE XI

Kohonen’s SOM For each input pattern/sample , the most similar neuron (considering its weight vector), and its neighbours , are activated. Working ALIFE XI The weight vector of every one of these neurons is updated to be closer to the input values. The SOM evolves , ‘moving’ the neurons in order to fit to the input space (of samples).

KohonAnts Presentation ALIFE XI It merges Ant System with Kohonen’s SOM concepts. It is a clustering and classification algorithm. * * INPUT SAMPLES Each input sample (vector of variables) is associated to an ant . They move in a toroidal grid which has a vector (of the same dimension) associated to each cell ( pheromone ) .

KohonAnts Working ALIFE XI Initialization Decide Where to Go For each Ant: Pheromone Update Pheromone Evaporation END Stop criteria is TRUE No more Ants Stop criteria is FALSE

KohonAnts Main Functions (I) ALIFE XI Decide Where to Go every ant evaluates its neighbourhood (similar to SOM) to find the cell where move to. A probability is associated to every cell . the probability of each cell depends on the euclidean distance between the ant vector and the centroid of a zone surrounding the cell. o o o o yellow circle is the neighbourhood (all possible cells to move) with radius = 2. green circles are areas to calculate centroid (marked with ’o’). the centroid is the arithmetic mean of the vectors inside the circle. So, every ant moves with higher probability to cells surrounded by vectors similar to its own .

KohonAnts Main Functions (II) ALIFE XI Pheromone Update once an ant chooses the cell to move to (inside its neighbourhood), it updates the vector of that cell and makes it closer to its own vector . it applies a formula similar to the learning function of SOM, so it depends on a learning rate  . Pheromone Evaporation once all the ants have moved, the pheromone tends to disappear (the vectors revert to its initial values). it uses a formula similar to the evaporation function of Ant Algorithms, which depends on an evaporation rate  .

KohonAnts Key Features ALIFE XI The neighbourhood radius and the learning rate decreases with the iterations (as in SOM). It means big changes at the beginning ( clustering ) and small changes the rest of the running ( refinement ). An ant can ‘jump’ or ‘fly’, since it can move to cells far than one hop . The pheromone changes the grid (environment) and makes it closer to the ant’s vector (input pattern/sample). The ants tend to form clusters near other similar ants (similar vectors). The environment (the grid) also ‘evolves’ and can be used as a classification tool .

Experiments Datasets ALIFE XI IRIS data of 3 species of Iris plant, 50 samples of each class, and 4 variables per sample (flower attributes). GLASS data of 6 classes of glass, 214 samples in total, and 9 variables per sample (chemical composition). PIMA data of 2 classes of diabetes patients, 768 samples , and 8 variables per sample (medical data).

Experiments Clustering (Study) ALIFE XI performed over IRIS plants dataset (150 samples, 3 classes) it shows ants’ final position in the grid after 100 iterations a clear line of behaviour can be marked depending on  and  it can be seen that clear clusters are formed for some values of these parameters

Experiments Clustering (Results) ALIFE XI performed over IRIS plants dataset (150 samples, 3 classes) it shows the evolution of the clusters of ants in a few iterations, some clusters appear then, they are improved/refined in the following iterations Initial situation Iteration 10 Iteration 40 Iteration 100

Experiments Classification ALIFE XI example performed over IRIS plants dataset (150 samples, 3 classes) this is the final situation of the grid pheromone. Each cell is associated to a class we apply the KNN method to classify, so the class associated to a sample is the correspondent to the (majority) of the K closest vectors (of the grid)

Experiments Classification (Results) ALIFE XI Every dataset has been transformed into 6: 3 of 50% training/50% test 3 of 90% training/10% test The results beat the classical KNN method In recent works also beat other methods such as Neural Networks, Fuzzy Logic, Linear Discriminant Analysis …

Conclusions And Future Wok ALIFE XI We have presented a new algorithm which combines ideas of two soft-computing methods such as Kohonen’s Self-Organizing Maps and Ant System Model . It can be used as a clustering and as a pattern classification method. The preliminary results yielded are very promising and beat some other techniques . FUTURE WORK : Make a comparison with some clustering algorithms and with other classification ones. Try more difficult problems. Implement and test some other ideas used in SOMs and AS which may be improve the algorithm. Such a neighbourhood updating, a stop criteria or weights in the decide where to go function.

Thank You! ALIFE XI The sources, bin and data can be found in: https://forja.rediris.es/websvn/wsvn/geneura/KohonAnts/

KohonAnts: A Self-Organizing Ant Algorithm for Clustering and Pattern Classification

More Related Content

What's hot (19)

Viewers also liked (9)

More from Antonio Mora (20)

Recently uploaded (20)

KohonAnts: A Self-Organizing Ant Algorithm for Clustering and Pattern Classification