Basic Learning Algorithms of ANN
Download as PPTX, PDF1 like668 views
The document summarizes five basic learning algorithms of artificial neural networks: Hebbian learning, memory-based learning, backpropagation, competitive learning, Adaline network, and Madaline network. It provides details on each algorithm, including mathematical formulas, steps involved, advantages and disadvantages, and applications.
Basic Learning Algorithms of ANN
- 1. Learning Algorithm of ANN By: Waseem Khan F/o Engg. & Technology Jamia Millia Islamia New Delhi-110025
- 2. INTRODUCTION Learning rules are algorithms which direct changes in the weights of the connections in a network. They are incorporating an error reduction procedure by employing the difference between the desired output and an actual output to change its weights during training.The learning rule is typically applied repeatedly to the same set of training inputs across a large number of epochs with error gradually reduced across epochs as the weights are fine-tuned.
- 3. FIVE BASIC LEARNING ALGORITHM OF ANN 1. Hebbian Learning 2. Memory Based Learning 3. Back propagation 4. Competitive learning 5. Adaline network 6. Madaline network
- 4. Hebbian Learning Hebbian Learning is a learning rule which is the oldest and most famous of all learning rules. Hebb’s principle can be described as a method of determining how to alter the weights between model neurons. The Hebbian rule mathematically: wkj (n)=F(yk(n), xj(n)) where F() is a function of both signals. The above formula can take many specific forms.
- 5. HEBBIAN RULE Typical examples are: Hebb’s hypothesis: In the simplest case we have just the product of the two signals (it is also called the activity product rule): wkj (n)=yk(n) xj(n) where is a learning rate. This form emphasizes the correlational nature of a Hebbian synapse.
- 6. HEBBIAN RULE Covariance hypothesis: In this case we replace the product of pre- and post-synaptic signals with the departure of of the same signals from their respective average values over a certain time interval. If x* and y* is their time-averaged value then the covariance form is defined by: wkj (n)=(yk(n)-y*) (xj(n)-x*)
- 7. Memory based Learning In memory-based learning, most of the past experiences are explicitly stored in a large memory of correctly classified input-output All memory-based learning algorithms involve two factors 1. Criterion 2. Learning rule applied in local neighbourhood
- 8. Advantages of Memory-Based Methods Lazy learning: o never need to learn a global model o many simple local models taken together can represent a more complex global model o better focussed learning o handles missing values, time varying distributions Very efficient cross-validation Intelligible learning method to many users Nearest neighbours support explanation and training
- 9. Weaknesses of Memory-Based Methods Curse of Dimensionality Run-time cost scales with training set size Large training sets will not fit in memory Many MBL methods are strict averages Sometimes doesn’t seem to perform as well as other methods such as neural nets Predicted values for regression not continuous
- 10. COMPETITIVE LEARNING In competitive learning, neurons compete among themselves to be activated. In this nodes compete for the right to respond to a subset of the input data. Competitive learning works by increasing the specialization of each node in the network. It is well suited to finding clusters within data.
- 11. N inputs units P output neurons P x N weights x1 x2 xN W11 W12 W22 WP1 WPN Y1 Y2 YP Pi N j jiji XWh ...2,1 1 01oriY
- 12. Three basic steps a set of neurons that are all the same a limit imposed on the strength of each neuron a mechanism that permits the neurons to compete- a winner-takes-all The standard competitive learning rule wkj = (xj-wkj) if neuron k wins the competition = 0 if neuron k loses the competition
- 13. ADALINE LEARNING Network with a single linear unit. It receives input from several units and also from one unit called bias. Uses bipolar activation for its input signals and its target output.
- 14. The total input received by the output neuron is given by Apply activation function over net input : Square of error
- 16. DEMERITS OF ADALINE Only for linearly separable problems Solves problems where have only one global minimum.
- 17. APLICATIONS OF ADALINE LEARNING Pattern Classifications Better convergence property than Perceptron Noise cancellation Echo cancellation Face recognition Signature recognition
- 18. MADALINE Combination of many Adalines. Architectures: Hidden layers of adaline nodes Output nodes differ Learning Error driven, but not by gradient descent Minimum disturbance: smaller change of weights is preferred, provided it can reduce the error
- 20. APLICATIONS OF MADALINE LEARNING Logical Calculation Signal Processing Vehicle inductive signature recognition Forecasting and risk assessments. Used in several adaptive filtering process. Used to solve three Monks problems, two Led display problems, and the And-Xor problem
- 21. ADVANTAGES OF MADALINE LEARNING Solves non-separable problems It is easy for description of discrete tasks without extra requirement of discretization Simple in computation and interpretation with hard-limit activation function and limited input and output states Facilitative for hardware implementation with the available VLSI technology
- 22. DISADVANTAGES OF MADALINE LEARNING Larger architecture Effect of variation of network parameters on its output Madaline sensitivity
- 23. BACK PROPAGATION Back propagation is a multilayer feed forward network with one layer of z hidden units. The y output units and z hidden units has b bias. The input layer is connected to hidden layer and output layer is connected to the output layer by means of interconnection weights.
- 25. MERITS OF BACK PROPAGATION Relatively simple implementation. It does not require any special mention of the features of the function to be learnt. Computing time is reduced if the weights chosen are small at the beginning. Batch updates of weights exist, which provides a smoothing effect on the weight correction terms.
- 26. DEMERITS OF BACKPROPAGATION Slow and inefficient. Large amount of input/output data is available. Outputs can be fuzzy or non numeric. The solutions of the problem may change over time within the bounds of given input and output parameters. Back propagation learning does not require normalization of input vectors; however, normalization could improve performance. Gradient descent with back propagation is not guaranteed to find the global minimum of the error function
- 27. APLICATIONS OF BACKPROPAGATION Load forecasting problems in power systems. Image processing. Fault diagnosis and fault detection. Gesture recognition, speech recognition. Signature verification. Bioinformatics. Structural engineering design (civil).