Artificial neural network
2 likes1,159 views
This document provides an overview of artificial neural networks (ANNs). It discusses how ANNs are inspired by biological neural networks and consist of interconnected artificial neurons that process information. The document describes common ANN architectures like multilayer perceptrons and radial basis function networks. It also summarizes different ANN learning paradigms such as supervised, unsupervised, and reinforcement learning. Specific learning rules and algorithms are mentioned, including the perceptron rule, Hebbian learning, competitive learning, and backpropagation. Applications of ANNs discussed include pattern recognition, clustering, prediction, and data compression.
![Network and Complex Systems www.iiste.org
ISSN 2224-610X (Paper) ISSN 2225-0603 (Online)
Vol.3, No.1, 2013-Selected from Inter national Conference on Recent Trends in Applied Sciences with Engineering Applications
27
either both positive or both negative at the same time have strong positive weights, while those that tend to be
opposite have strong negative weights.
Wi(new)=Wi(old)+XiY
For example, we have heard the word Nokia for many years, and are still hearing it. We are pretty used to
hearing Nokia Mobile Phones, i.e. the word 'Nokia' has been associated with the word 'Mobile Phone' in our
Mind. Every time we see a Nokia Mobile, the association between the two words 'Nokia' and 'Phone' gets
strengthened in our mind. The association between 'Nokia' and 'Mobile Phone' is so strong that if someone tried
to say Nokia is manufacturing Cars and Trucks, it would seem odd.
Competitive Wing Rules
Unlike Hebbian learning, competitive-learning output units compete among themselves for activation. As a
result, only one output unit is active at any given time. This phenomenon is known as winner-take-all.
Competitive learning has beenfound to exist in biological neural network.Competitive learning often clusters or
categorizes the input data. Similar patterns are grouped by the network and represented by a single unit. This
grouping is done automatically based on data correlations. The simplest competitive learning network consists of
a single layer of output units.
[Note: Only the weights of the winner unit get updated.]
The effect of this learning rule is to move the stored pattern in the winner unit (weights) a little bit closer to the
input pattern. From the competitive learning rule we can see that the network will not stop learning (updating
weights) unless the learning rate 0. A particular input pattern can fire different output units at different iterations
during learning. This brings up the stability issue of a learning system. The system is said to be stable if no
pattern in the training data changes its category after a finite number of learning iterations. One way to achieve
stability is to force the learning rate to decrease gradually as the learning process proceeds .
However, this artificial freezing of learning causes another problem termed plasticity, which is the
ability to adapt to new data. This is known as Grossberg’s stability plasticity dilemma in competitive learning.
Eg: Vector quantization for data compression: It has been widely used in speech and image processing for
efficient storage, transmission, and modeling. Its goal is to represent a set or distribution of input vectors with a
relatively small number of prototype vectors (weight vectors), or a codebook. Once a codebook has been
constructed and agreed upon by both the transmitter and the receiver, you need only transmit or store the index
of the corresponding prototype to the input vector.
The Back propagation Algorithm
The back propagation algorithm (Rumelhart and McClelland, 1986) is used in layered feed-forward ANNs. This
means that the artificial neurons are organized in layers, and send their signals “forward”, and then the errors are
propagated backwards. The network receives inputs by neurons in the input layer, and the output of the network
is given by the neurons on an output layer. There may be one or more intermediate hidden layers. The
backpropagation algorithm uses supervised learning. After finding error, we can adjust the weights using the
method of gradient descendent:
(formule of back prop algo)
For practical reasons, ANNs implementing the backpropagation algorithm do not have too many layers, since the
time for training the networks grows exponentially. Also, there are refinements to the backpropagation algorithm
which allow a faster learning.
5. APPLICATIONS
Although successful applications can be found in certain well-constrained environments, none is flexible enough
to perform well outside its domain. ANNs provide exciting alternatives, and many applications could benefit
from using them. Modern digital computers outperform humans in the domain of numeric computation and
related symbol manipulation.
Air traffic control could be automated with the location,altitude,direction and speed of each radar blip taken
as input to the network. The would be the air traffic controller’s instruction in response to each blip.
Animal behavior and population cycles may be suitable for analysis by neural networks. Appraisal and
valuation of property, building,automobiles…etc should be ab easy task for a neural network.
Echo pattern from sonar,radar and magnetic instruments could be used to predict their targets.
Weather prediction may be possible.
Voice recognization could be obtained by analyzing audio oscilloscope pattern.
Traffic flows could be predicted so that signal timing could be optimized…..many more.
CONCLUTION
Using artificial neural networks modern digital computers outperform humans in the domain of numeric](https://image.slidesharecdn.com/artificialneuralnetwork-130614011610-phpapp01/85/Artificial-neural-network-4-320.jpg)
Artificial neural network
- 1. Network and Complex Systems www.iiste.org ISSN 2224-610X (Paper) ISSN 2225-0603 (Online) Vol.3, No.1, 2013-Selected from Inter national Conference on Recent Trends in Applied Sciences with Engineering Applications 24 Artificial Neural Network Neha Gupta Institute of Engineerinf and Technology,DAVV,Indore Email:-nehaguptaneema@gmail.com ABSTRACT The long course of evolution has given the human brain many desirable characteristics not present in Von Neumann or modern parallel computers. These include massive parallelism, distributed representation and computation, learning ability, generalization ability,adaptivity, inherent contextual information processing, fault tolerance, and low energy consumption. It is hoped that devices based on biological neural networks will possess some of these desirable characteristics.On this basic we come out with the concept of artificial neural network. An artificial neural network, often just called a neural network, is a mathematical model inspired by biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes information using a connectionist approach to computation. Neural networks have emerged in the past few years as an area of unusual opportunity for research, development and application to a variety of real world problems. Indeed, neural networks exhibit characteristics and capabilities not provided by any other technology. The article discusses the motivations behind the development of ANNs and describes the basic biological neuron. This paper presents a brief tutorial on artificial neural networks, some of the most commonly used ANN models and briefly describes several applications of it. 1. INTRODUCTION Inspired by biological neural networks, ANNs are massively parallel computing systems consisting of an exremely large number of simple processors with many interconnections. ANN models attempt to use some “organizational” principles believed to be used in the human. One type of network sees the nodes as ‘artificial neurons’. These are called artificial neural networks (ANNs). An artificial neuron is a computational model inspired in the natural neurons. Since the function of ANNs is to process information, they are used mainly in fields related with it. There are a wide variety of ANNs that are used to model real neural networks, and study behaviour and control in animals and machines, but also there are ANNs which are used for engineering purposes, such as pattern recognition, forecasting,and data compression. These basically consist of inputs (like synapses), which are multiplied by weights. Weights assigned with each arrow represent information flow. These weights are then computed by a mathematical function which determines the activation of the neuron. Another function (which may be the identity) computes the output of the artificial neuron (sometimes in dependence of a certain threshold). The neurons of this network just sum their inputs. Since the input neurons have only one input, their output will be the input they received multiplied by a weight. Biological Neuron
- 2. Network and Complex Systems ISSN 2224-610X (Paper) ISSN 2225-0603 (Online Vol.3, No.1, 2013-Selected from Inter national Conference on If the weight is high,then input will be strong. By adjusting the weights of an artificial neuron we can obtain the output we want for specific inputs. But when we have an ANN of hundreds or thousands of neurons, it would be quite complicated to find by hand all the necessary weights. But we can find algorithms which can adjust the weights of the ANN in order to obtain the desired output from the network. This process of adjusting the weights is called learning or training. . The training begins with random error will be minimal. 2.CHALLENGING PROBLEMS a)Pattern recognization: Pattern recognition is the assignment of a label to a given input value. Well known application include Other typical applications of pattern recognition techniques are automatic spam email messages), the automatic recognition of handwritten postal recognition of images of human faces, or handwriting image extraction from medical forms. b)Clustering/Categorization: Categorization is the process in which ideas and objects are and understood. Categorization implies that objects are grouped into categories, purpose.applications of clustering include data mining,data compression and exploratory data analysis. c)Prediction/Forecasting: It has a significant impacton decition making in bussiness,science and engineering. Stock market weather forecasting are typical application of prediction/forecasting techniques. 3. NETWORK ARCHITECTURE Based on the connection pattern network architecture can be grouped into two categories : • Feed-forward networks, in which graphs have • Recurrent (or feedback) networks, in which loops occur because of feedback connections. Generally,Feed-Forward Networks sequence of values from a given input. Feedforwar an input is independent of the previous network state. Multilayer Perceptron Architecture: network. “Fully connected” means that the output from each input and hidden neuron is distributed to all of the neurons in the following layer. All neural networks have an input layer and an output layer, but the number of hidden layers may vary. Here is a diagram of a perceptron ne When there is more than one hidden layer, the output from one hidden layer is fed into the next hidden layer and separate weights are applied to the sum going into each layer. Radial Basis Function network: A basis functions as activation functions functions of the inputs and neuron parameters. Radial basis function networks are used for approximation, time series prediction (Online) Inter national Conference on Recent Trends in Applied Sciences with Engineering Applications 25 Artificial Neuron If the weight is high,then input will be strong. By adjusting the weights of an artificial neuron we can obtain the output we want for specific inputs. But when we have an ANN of hundreds or thousands of neurons, it would be nd all the necessary weights. But we can find algorithms which can adjust the weights of the ANN in order to obtain the desired output from the network. This process of adjusting the weights . . The training begins with random weights, and the goal is to adjust them so that the is the assignment of a label to a given input value. Well known application include Other of pattern recognition techniques are automatic recognition, classification automatic recognition of handwritten postal codes on postal envelopes, of human faces, or handwriting image extraction from medical forms. in which ideas and objects are recognized Categorization implies that objects are grouped into categories, usually for some specific purpose.applications of clustering include data mining,data compression and exploratory data analysis. It has a significant impacton decition making in bussiness,science and engineering. Stock market weather forecasting are typical application of prediction/forecasting techniques. 3. NETWORK ARCHITECTURE Based on the connection pattern network architecture can be grouped into two categories : networks, in which graphs have no loops. networks, in which loops occur because of feedback connections. Forward Networks are static,that is, they produce only one set of output values sequence of values from a given input. Feedforward networks are memory-less in the sense that their response to an input is independent of the previous network state. Multilayer Perceptron Architecture: This is a full-connected, three layer, feed-forward, perceptron neural means that the output from each input and hidden neuron is distributed to all of the neurons in the following layer. All neural networks have an input layer and an output layer, but the number of hidden layers may vary. Here is a diagram of a perceptron network with two hidden layers and four total layers: When there is more than one hidden layer, the output from one hidden layer is fed into the next hidden layer and separate weights are applied to the sum going into each layer. A radial basis function network is an artificial neural network activation functions. The output of the network is a linear combination functions of the inputs and neuron parameters. Radial basis function networks are used for time series prediction, and system control. www.iiste.org Recent Trends in Applied Sciences with Engineering Applications If the weight is high,then input will be strong. By adjusting the weights of an artificial neuron we can obtain the output we want for specific inputs. But when we have an ANN of hundreds or thousands of neurons, it would be nd all the necessary weights. But we can find algorithms which can adjust the weights of the ANN in order to obtain the desired output from the network. This process of adjusting the weights weights, and the goal is to adjust them so that the is the assignment of a label to a given input value. Well known application include Other recognition, classification (e.g. spam/non- on postal envelopes, automatic of human faces, or handwriting image extraction from medical forms. recognized, differentiated, usually for some specific purpose.applications of clustering include data mining,data compression and exploratory data analysis. It has a significant impacton decition making in bussiness,science and engineering. Stock market prediction and Based on the connection pattern network architecture can be grouped into two categories : networks, in which loops occur because of feedback connections. that is, they produce only one set of output values rather than a less in the sense that their response to forward, perceptron neural means that the output from each input and hidden neuron is distributed to all of the neurons in the following layer. All neural networks have an input layer and an output layer, but the number of twork with two hidden layers and four total layers: When there is more than one hidden layer, the output from one hidden layer is fed into the next hidden layer and artificial neural network that uses radial linear combination of radial basis functions of the inputs and neuron parameters. Radial basis function networks are used for function
- 3. Network and Complex Systems www.iiste.org ISSN 2224-610X (Paper) ISSN 2225-0603 (Online) Vol.3, No.1, 2013-Selected from Inter national Conference on Recent Trends in Applied Sciences with Engineering Applications 26 Issues: There are many issues in designing feed-forward networks: How many layers are needed for a given task, How many units are needed per layer, How will the network perform on data not included in the training set (generalization ability), and How large the training set should be for “good” generalization. Although multilayer feed-forward networks using back propagation have been widely employed for classification and function approximation. Recurrent, or feedback, networks, on the other hand, are dynamic systems. When a new input pattern is presented, the neuron outputs are computed. Because of the feedback paths, the inputs to each neuron are then modified, which leads the network to enter a new state. Different network architectures require appropriate learning algorithms. 4. OVERVIEW OF LEARNING PROCESSES: A learning process in the ANN context can be viewed as the problem of updating network architecture and connection weights so that a network can efficiently perform a specific task. The network usually must learn the connection weights from available training patterns. Performance is improved over time by iteratively updating the weights in the network. ANNs' ability to automatically learn from examples makes them attractive and exciting. Instead of following a set of rules specified by human experts, ANNs appear to learn underlying rules (like input-output relationships) from the given collection of representative examples. This is one of the major advantages of neural networks over traditional expert systems. Learning paradigms: There are three major learning paradigms, each corresponding to a particular abstract learning task.These are supervised learning, unsupervised learning and reinforcement learning. Supervised learning the network is provided with a correct answer (output) for every input pattern. Weights are determined to allow the network to produce answers as close as possible to the known correct answers. Unsupervised learning does not require a correct answer associated with each input pattern in the training data set. It explores the underlying structure in the data, or correlations between patterns in the data, and organizes patterns into categories from these correlations. Hybrid learning combines supervised and unsupervised learning. Part of the weights are usually determined through supervised learning, while the others are obtained through unsupervised learning. Reinforcement learning is a variant of supervised learning in which the network is provided with only a critique on the correctness of network outputs, not the correct answers themselves. Error-Correction Rules: In the supervised learning paradigm, the network is given a desired output for each input pattern. During the learning process, the actual output generated by the network may not be equal to the desired output. The basic principle of error-correction learning rules is to use the error signal (target output-obtained output) to modify the connection weights to gradually reduce this error. Perceptron learning rule: The perceptron is an algorithm for supervised classification of an input into one of several possible non-binary outputs. The learning algorithm for perceptrons is an online algorithm, in that it processes elements in the training set one at a time. In the context of artificial neural networks, a perceptron is similar to a linear neuron except that it does classification rather than regression. That is, where a perceptron adjusts its weights to separate the training examples into their respective classes, a linear neuron adjusts its weights to reduce a real- valued prediction error. The perceptron algorithm is also termed the single-layer perceptron, to distinguish it from a multilayer perceptron, which is a misnomer for a more complicated neural network. As a linear classifier, the single-layer perceptron is the simplest feedforward neural network. Boltzmann learning Boltzmann learning is derived from information-theoretic and thermodynamic principles.The objective of Boltzmann learning is to adjust the connection weights so that the states of visible units satisfy a particular desired probability distribution. Boltzmann learning can be viewed as a special case of error-correction learning in which error is measured not as the direct difference between desired and actual outputs, but as the difference between the correlations among the outputs of two neurons under clamped and free running operating conditions. Hebbian Rule The theory is often summarized as "Cells that fire together, wire together." Hebb's principle can be described as a method of determining how to alter the weights between model neurons. The weight between two neurons increases if the two neurons activate simultaneously and reduces if they activate separately. Nodes that tend to be
- 4. Network and Complex Systems www.iiste.org ISSN 2224-610X (Paper) ISSN 2225-0603 (Online) Vol.3, No.1, 2013-Selected from Inter national Conference on Recent Trends in Applied Sciences with Engineering Applications 27 either both positive or both negative at the same time have strong positive weights, while those that tend to be opposite have strong negative weights. Wi(new)=Wi(old)+XiY For example, we have heard the word Nokia for many years, and are still hearing it. We are pretty used to hearing Nokia Mobile Phones, i.e. the word 'Nokia' has been associated with the word 'Mobile Phone' in our Mind. Every time we see a Nokia Mobile, the association between the two words 'Nokia' and 'Phone' gets strengthened in our mind. The association between 'Nokia' and 'Mobile Phone' is so strong that if someone tried to say Nokia is manufacturing Cars and Trucks, it would seem odd. Competitive Wing Rules Unlike Hebbian learning, competitive-learning output units compete among themselves for activation. As a result, only one output unit is active at any given time. This phenomenon is known as winner-take-all. Competitive learning has beenfound to exist in biological neural network.Competitive learning often clusters or categorizes the input data. Similar patterns are grouped by the network and represented by a single unit. This grouping is done automatically based on data correlations. The simplest competitive learning network consists of a single layer of output units. [Note: Only the weights of the winner unit get updated.] The effect of this learning rule is to move the stored pattern in the winner unit (weights) a little bit closer to the input pattern. From the competitive learning rule we can see that the network will not stop learning (updating weights) unless the learning rate 0. A particular input pattern can fire different output units at different iterations during learning. This brings up the stability issue of a learning system. The system is said to be stable if no pattern in the training data changes its category after a finite number of learning iterations. One way to achieve stability is to force the learning rate to decrease gradually as the learning process proceeds . However, this artificial freezing of learning causes another problem termed plasticity, which is the ability to adapt to new data. This is known as Grossberg’s stability plasticity dilemma in competitive learning. Eg: Vector quantization for data compression: It has been widely used in speech and image processing for efficient storage, transmission, and modeling. Its goal is to represent a set or distribution of input vectors with a relatively small number of prototype vectors (weight vectors), or a codebook. Once a codebook has been constructed and agreed upon by both the transmitter and the receiver, you need only transmit or store the index of the corresponding prototype to the input vector. The Back propagation Algorithm The back propagation algorithm (Rumelhart and McClelland, 1986) is used in layered feed-forward ANNs. This means that the artificial neurons are organized in layers, and send their signals “forward”, and then the errors are propagated backwards. The network receives inputs by neurons in the input layer, and the output of the network is given by the neurons on an output layer. There may be one or more intermediate hidden layers. The backpropagation algorithm uses supervised learning. After finding error, we can adjust the weights using the method of gradient descendent: (formule of back prop algo) For practical reasons, ANNs implementing the backpropagation algorithm do not have too many layers, since the time for training the networks grows exponentially. Also, there are refinements to the backpropagation algorithm which allow a faster learning. 5. APPLICATIONS Although successful applications can be found in certain well-constrained environments, none is flexible enough to perform well outside its domain. ANNs provide exciting alternatives, and many applications could benefit from using them. Modern digital computers outperform humans in the domain of numeric computation and related symbol manipulation. Air traffic control could be automated with the location,altitude,direction and speed of each radar blip taken as input to the network. The would be the air traffic controller’s instruction in response to each blip. Animal behavior and population cycles may be suitable for analysis by neural networks. Appraisal and valuation of property, building,automobiles…etc should be ab easy task for a neural network. Echo pattern from sonar,radar and magnetic instruments could be used to predict their targets. Weather prediction may be possible. Voice recognization could be obtained by analyzing audio oscilloscope pattern. Traffic flows could be predicted so that signal timing could be optimized…..many more. CONCLUTION Using artificial neural networks modern digital computers outperform humans in the domain of numeric
- 5. Network and Complex Systems www.iiste.org ISSN 2224-610X (Paper) ISSN 2225-0603 (Online) Vol.3, No.1, 2013-Selected from Inter national Conference on Recent Trends in Applied Sciences with Engineering Applications 28 computation and related symbol manipulation. REFERENCE “Principles Of Soft Computing” by S. N. Sivanandam,S. N. Deepa,(2008). “Soft Computing And Intelligent System Design-Theory Tools And Application” by Fakhreddine Karray and Clarence De Silva (2004). “Neural Network- A Comprehensive Foundation” by Simon S. Haykin. (1999). “An Introduction To Neural Network”, by James A. Anderson,(1997). “Fundamentals Of Neural Network” by Laurene V. Fausett,(1993). Ash, T. (1989), ‘Dynamic node creation in backpropagation neural networks’, Connection Science 1(4), 365–375. Jain,Anil K. Michigan State Univ., East Lansing,MI,USA Mao,Jianchang; Mohiuddin,K.M. (1996) Related documents from internet.
- 6. This academic article was published by The International Institute for Science, Technology and Education (IISTE). The IISTE is a pioneer in the Open Access Publishing service based in the U.S. and Europe. The aim of the institute is Accelerating Global Knowledge Sharing. More information about the publisher can be found in the IISTE’s homepage: http://www.iiste.org CALL FOR PAPERS The IISTE is currently hosting more than 30 peer-reviewed academic journals and collaborating with academic institutions around the world. There’s no deadline for submission. Prospective authors of IISTE journals can find the submission instruction on the following page: http://www.iiste.org/Journals/ The IISTE editorial team promises to the review and publish all the qualified submissions in a fast manner. All the journals articles are available online to the readers all over the world without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. Printed version of the journals is also available upon request of readers and authors. IISTE Knowledge Sharing Partners EBSCO, Index Copernicus, Ulrich's Periodicals Directory, JournalTOCS, PKP Open Archives Harvester, Bielefeld Academic Search Engine, Elektronische Zeitschriftenbibliothek EZB, Open J-Gate, OCLC WorldCat, Universe Digtial Library , NewJour, Google Scholar