SlideShare a Scribd company logo

Introduction to Neural networks (under graduate course) Lecture 5 of 9

Download as PPTX, PDF
Randa Elanwar, profile picture
Randa Elanwar

This document presents a lecture on linearly separable functions in neural networks, specifically focusing on the perceptron learning rule and various methods to solve 2-class problems. It details the process of updating weight vectors in both pattern and batch modes, including iterations and convergence examples. Graphical representation of class separation, along with the influence of learning rates and biases, is also discussed.

Education
Related topics:
Neural Networks
1 of 22
Downloaded 74 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Neural Networks Dr. Randa Elanwar Lecture 5
Lecture Content • Linearly separable functions: 2-class problem implementation – Learning laws: Perceptron learning rule – Pattern mode solution method – Batch mode solution method 2Neural Networks Dr. Randa Elanwar
Linearly Separable Functions • Example: logical OR, with initial weights 0.5, 0.3 with bias = 0.5 and activation step function at t=0.5. The learning rate = 1 3Neural Networks Dr. Randa Elanwar x2 w1= 0.5 w2 = 0.3 x1 yin = x1w1 + x2w2   y Activation Function: Binary Step Function t = 0.5, (y-in) = 1 if y-in >= t otherwise (y-in) = 0
Solving Linearly Separable Functions (Pattern mode) • Given: • Since we consider bias as additional weight thus the weight vector is 1x3 we have to fix the dimensionality of the input vector x1, x2, x3 and x4 from 2x1 to be 3x1 to perform the multiplication. 4Neural Networks Dr. Randa Elanwar            11 10 01 00 X).( bXWfY  x1 x2 x3 x4 x1 x2 y 0 0 0 0 1 1 1 0 1 1 1 1           111 101 011 001 X  5.03.05.0)0( W
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 1 5Neural Networks Dr. Randa Elanwar OK Wrong 1,1 1 0 1 ].5.03.15.0[3.)1(           yXW 1,3.2 1 1 1 ].5.03.15.0[4.)1(           yXW OK OK Wrong1,5.0 1 0 0 ].5.03.15.0[1.)1(           yXW   0,5.0 1 0 0 .5.03.05.01.)0(           yXW   0,2.0 1 1 0 .5.03.05.02.)0(           yXW          5.0 3.1 5.0 )..( 2)0()1( XyWW ydis TT 
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 2 • Update weight vector for iteration 3 6Neural Networks Dr. Randa Elanwar           5.0 3.1 5.0 1)..()1()2( XyyWW dis TT  1,8.0 1 1 0 ].5.03.15.0[2.)2(           yXW 0,0 1 0 1 ].5.03.15.0[3.)2(           yXW OK Wrong          5.0 3.1 5.1 3)..()2()3( XyyWW dis TT  1,5.0 1 0 0 ].5.03.15.1[1.)3(           yXW 1,3.3 1 1 1 ].5.03.15.1[4.)3(           yXW OK Wrong
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 4 • The weights learning has converged at 4 iterations 7Neural Networks Dr. Randa Elanwar 0,5.0 1 0 0 ].5.03.15.2[1.)4(           yXW 1,8.0 1 1 0 ].5.03.15.1[2.)4(           yXW           5.0 3.1 5.1 1)..()3()4( XyyWW dis TT  1,1 1 0 1 ].5.03.15.1[3.)4(           yXW 1,3.2 1 1 1 ].5.03.15.1[4.)4(           yXW OK OK OK OK
Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 1 • Add w for all misclassified inputs together in 1 step 8Neural Networks Dr. Randa Elanwar 0,0 1 0 1 ].5.03.05.0[3.)0(           yXW 0,3.0 1 1 1 ].5.03.05.0[4.)0(           yXW OK Wrong Wrong Wrong 4)..(3)..(2)..()0()1( XyXyXy yyyWW disdisdis TT                                                 5.2 3.2 5.2 1 1 1 1 0 1 1 1 0 5.0 3.0 5.0 )1(W T   0,5.0 1 0 0 .5.03.05.01.)0(           yXW   0,2.0 1 1 0 .5.03.05.02.)0(           yXW
Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 1 • Add w for all misclassified inputs together in 1 step 9Neural Networks Dr. Randa Elanwar 1,5.2 1 0 0 ].5.23.25.2[1.)1(           yXW 1,8.4 1 1 0 ].5.23.25.2[2.)1(           yXW 1,5 1 0 1 ].5.23.25.2[3.)1(           yXW 1,3.7 1 1 1 ].5.23.25.2[4.)1(           yXW Wrong OK OK OK          5.1 3.2 5.2 1)..()1()2( XyyWW dis TT 
Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 3 • Add w for all misclassified inputs together in 1 step 10Neural Networks Dr. Randa Elanwar 1,5.1 1 0 0 ].5.13.25.2[1.)2(           yXW 1,8.3 1 1 0 ].5.13.25.2[2.)2(           yXW 1,4 1 0 1 ].5.13.25.2[3.)2(           yXW 1,3.6 1 1 1 ].5.13.25.2[4.)2(           yXW OK OK OK Wrong          5.0 3.2 5.2 1)..()2()3( XyyWW dis TT 
Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 4 • Add w for all misclassified inputs together in 1 step 11Neural Networks Dr. Randa Elanwar OK OK OK Wrong           5.0 3.2 5.2 1)..()3()4( XyyWW dis TT  1,5.0 1 0 0 ].5.03.25.2[1.)3(           yXW 1,8.2 1 1 0 ].5.03.25.2[2.)3(           yXW 1,3 1 0 1 ].5.03.25.2[3.)3(           yXW 1,3.5 1 1 1 ].5.03.25.2[4.)3(           yXW
Solving Linearly Separable Functions (Batch mode) • The weights learning has also converged at 4 iterations but with different final values 12Neural Networks Dr. Randa Elanwar OK OK OK OK0,5.0 1 0 0 ].5.03.25.2[1.)4(           yXW 1,8.1 1 1 0 ].5.03.25.2[2.)4(           yXW 1,2 1 0 1 ].5.03.25.2[3.)4(           yXW 1,3.4 1 1 1 ].5.03.25.2[4.)4(           yXW
Linearly Separable Functions • Example: Consider linearly separable patterns where class C1 consists of the two patterns [1 0]T and [1 1]T and C2 consists of the two patterns [0 0]T and [0 1]T. Use the perceptron algorithm with  = 1 and w(0)= [0 1 -1/2]T to design the line separating the two classes. 13Neural Networks Dr. Randa Elanwar X2 X1 X4 X3 X2 X1 initial finalIt is very important to graph the problem to define the initial line and assign the direction of positive and negative Initial weight  X2-1/2 = 0, intersects vertical axis at (0,1/2) and is parallel to horizontal axis When X2>1/2 we get +ve value thus the positive direction is above the line +ve -ve
Linearly Separable Functions • Initially x2 and x3 are in the correct side while x1 and x4 are in the wrong side. • Thus we assume x1 and x2 have to be on the +ve side and x3 and x4 have to be on the –ve side • Note that sometimes you are not given the activation function f. In such case: you can compute outputs and update weights by polarity (sign) instead of the activation function value: If the W.X >0 and it is wrong w = .(-1).X If the W.X <0 and it is wrong w = .(+1).X 14Neural Networks Dr. Randa Elanwar
Solving Linearly Separable Functions (Pattern mode) • Again since the weight vector is 1x3 we have to fix the dimensionality of the input vector x1, x2, x3 and x4 from 2x1 to be 3x1 to perform the multiplication. • Update weight vector for iteration 1 15Neural Networks Dr. Randa Elanwar 5.0 1 0 1 ].5.010[1.)0(          XW -ve and it have to be +ve          5.0 1 1 1)0()0()1( Xw WWW TTT 5.2 1 1 1 ].5.011[2.)1(          XW OK 5.0 1 0 0 ].5.011[3.)1(          XW +ve and it have to be -ve
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 2 • Update weight vector for iteration 3 • Update weight vector for iteration 4 16Neural Networks Dr. Randa Elanwar           5.0 1 1 3)1()1()2( Xw WWW TTT 5.0 1 1 0 ].5.011[4.)2(          XW +ve and it have to be -ve           5.1 0 1 4)2()2()3( Xw WWW TTT 5.0 1 0 1 ].5.101[1.)3(          XW -ve and it have to be +ve           5.0 0 2 1)3()3()4( Xw WWW TTT 5.1 1 1 1 ].5.002[2.)4(          XW OK
Solving Linearly Separable Functions (Pattern mode) • The new straight line equation is 2X1-1/2 = 0 a vertical line intersecting the horizontal axis at 1/4 and is parallel to the vertical axis • The solution has converged in 4 iterations. 17Neural Networks Dr. Randa Elanwar   5.0 1 0 0 5.0023.)4(          XW OK   5.0 1 1 0 5.0024.)4(          XW OK   5.1 1 0 1 5.0021.)4(          XW OK
Solving Linearly Separable Functions (Batch mode) • Again since the weight vector is 3x1 we have to fix the dimensionality of the input vector x1, x2, x3 and x4 to be 1x3 to perform the multiplication. • Update weight vector for iteration 1. Add w for all misclassified inputs together in 1 step 18Neural Networks Dr. Randa Elanwar +ve and it have to be -ve -ve and it have to be +ve OK OK 4.1.)0()1( XXWW TT                                         5.0 0 1 1 1 0 1 0 1 5.0 1 0 )1(W T 5.0 1 0 1 ].5.010[1.)0(          XW 1 1 1 1 ].5.010[2.)0(          XW 5.0 1 0 0 ].5.010[3.)0(          XW 5.0 1 1 0 ].5.010[4.)0(          XW
Solving Linearly Separable Functions (Batch mode) • The new straight line equation is X1-1/2 = 0 a vertical line intersecting the horizontal axis at 1/2 and is parallel to the vertical axis • The solution has converged in 1 iteration. 19Neural Networks Dr. Randa Elanwar OK OK OK OK5.0 1 0 1 ].5.001[1.)1(          XW 5.0 1 1 1 ].5.001[2.)1(          XW 5.0 1 0 0 ].5.001[3.)1(          XW 5.0 1 1 0 ].5.001[4.)1(          XW
Non linear problems • XOR problem • No way to draw a line to separate the positive from negative examples 20Neural Networks Dr. Randa Elanwar Input1 Input2 Output 0 0 0 0 1 1 1 0 1 1 1 0
Non linear problems • XOR problem • The only way to separate the positive from negative examples is to draw 2 lines (i.e., we need 2 straight line equations) or nonlinear region to capture one type only 21Neural Networks Dr. Randa Elanwar +ve +ve -ve -ve+ve -ve cba yx  22
Non linear problems • To implement the nonlinearity we need to insert one or more extra layer of nodes between the input layer and the output layer (Hidden layer) 22Neural Networks Dr. Randa Elanwar

More Related Content

PPTX
Introduction to Neural networks (under graduate course) Lecture 4 of 9
Randa Elanwar
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 6 of 9
Randa Elanwar
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 8 of 9
Randa Elanwar
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 7 of 9
Randa Elanwar
 
PPTX
03 Single layer Perception Classifier
Tamer Ahmed Farrag, PhD
 
PPTX
Machine Learning - Neural Networks - Perceptron
Andrew Ferlitsch
 
PDF
Artificial neural networks
arjitkantgupta
 
PPTX
Unit 1
Vinod Srinivasan
 
Introduction to Neural networks (under graduate course) Lecture 4 of 9
Randa Elanwar
 
Introduction to Neural networks (under graduate course) Lecture 6 of 9
Randa Elanwar
 
Introduction to Neural networks (under graduate course) Lecture 8 of 9
Randa Elanwar
 
Introduction to Neural networks (under graduate course) Lecture 7 of 9
Randa Elanwar
 
03 Single layer Perception Classifier
Tamer Ahmed Farrag, PhD
 
Machine Learning - Neural Networks - Perceptron
Andrew Ferlitsch
 
Artificial neural networks
arjitkantgupta
 
Unit 1
Vinod Srinivasan
 

What's hot (20)

PDF
Introduction to Artificial Neural Networks
Stratio
 
PPT
Multilayer perceptron
smitamm
 
PPTX
Fuzzy and nn
Shimi Haridasan
 
PPTX
Artificial neural network
Ildar Nurgaliev
 
PPTX
Neural Networks
Sagacious IT Solution
 
PPT
lecture07.ppt
butest
 
PPSX
Perceptron (neural network)
EdutechLearners
 
PPTX
04 Multi-layer Feedforward Networks
Tamer Ahmed Farrag, PhD
 
PPT
Artificial Neural Networks
Arslan Zulfiqar
 
PPT
Lec 3-4-5-learning
Taymoor Nazmy
 
PPTX
Deep neural networks & computational graphs
Revanth Kumar
 
PPTX
Basic Learning Algorithms of ANN
waseem khan
 
PPTX
Deep learning algorithms
Revanth Kumar
 
PDF
Introduction to Artificial Neural Network
Qingkai Kong
 
PPTX
Activation function
RakshithGowdakodihal
 
PDF
Artificial Neural Network Lect4 : Single Layer Perceptron Classifiers
Mohammed Bennamoun
 
PPT
Soft Computering Technics - Unit2
sravanthi computers
 
PPT
Hebbian Learning
ESCOM
 
PDF
neural networksNnf
Sandilya Sridhara
 
PPTX
Backpropagation And Gradient Descent In Neural Networks | Neural Network Tuto...
Simplilearn
 
Introduction to Artificial Neural Networks
Stratio
 
Multilayer perceptron
smitamm
 
Fuzzy and nn
Shimi Haridasan
 
Artificial neural network
Ildar Nurgaliev
 
Neural Networks
Sagacious IT Solution
 
lecture07.ppt
butest
 
Perceptron (neural network)
EdutechLearners
 
04 Multi-layer Feedforward Networks
Tamer Ahmed Farrag, PhD
 
Artificial Neural Networks
Arslan Zulfiqar
 
Lec 3-4-5-learning
Taymoor Nazmy
 
Deep neural networks & computational graphs
Revanth Kumar
 
Basic Learning Algorithms of ANN
waseem khan
 
Deep learning algorithms
Revanth Kumar
 
Introduction to Artificial Neural Network
Qingkai Kong
 
Activation function
RakshithGowdakodihal
 
Artificial Neural Network Lect4 : Single Layer Perceptron Classifiers
Mohammed Bennamoun
 
Soft Computering Technics - Unit2
sravanthi computers
 
Hebbian Learning
ESCOM
 
neural networksNnf
Sandilya Sridhara
 
Backpropagation And Gradient Descent In Neural Networks | Neural Network Tuto...
Simplilearn
 
Ad

Viewers also liked (9)

PPTX
fuzzy logic application
Ankur Mahajan
 
PPTX
Swaroop.m.r
rangaishenvimilind
 
PPTX
Short course fuzzy logic applications
Dalia Baziuke
 
PPTX
Fuzzy logic and application in AI
Ildar Nurgaliev
 
PPTX
Fuzzy logic application (aircraft landing)
Piyumal Samarathunga
 
PPT
Fuzzy logic
Babu Appat
 
PPTX
Chapter 5 - Fuzzy Logic
Ashique Rasool
 
PPTX
Application of fuzzy logic
Viraj Patel
 
PPT
Fuzzy logic ppt
Priya_Srivastava
 
fuzzy logic application
Ankur Mahajan
 
Swaroop.m.r
rangaishenvimilind
 
Short course fuzzy logic applications
Dalia Baziuke
 
Fuzzy logic and application in AI
Ildar Nurgaliev
 
Fuzzy logic application (aircraft landing)
Piyumal Samarathunga
 
Fuzzy logic
Babu Appat
 
Chapter 5 - Fuzzy Logic
Ashique Rasool
 
Application of fuzzy logic
Viraj Patel
 
Fuzzy logic ppt
Priya_Srivastava
 
Ad

Similar to Introduction to Neural networks (under graduate course) Lecture 5 of 9 (20)

PPTX
CS767_Lecture_04.pptx
ShujatHussainGadi
 
PPTX
Neural Networks
Makerere Unversity School of Public Health, Victoria University
 
PPTX
Artificial neural networks - A gentle introduction to ANNS.pptx
AttaNox1
 
PPTX
Machine learning with neural networks
Let's talk about IT
 
PPTX
ANN presentation explaination and architecture.pptx
Account1850
 
PDF
Artificial Neural Networks Lect2: Neurobiology & Architectures of ANNS
Mohammed Bennamoun
 
PDF
ARTIFICIAL-NEURAL-NETWORKMACHINELEARNING
mohanapriyastp
 
PDF
Chapter8 LINEAR DESCRIMINANT FOR MACHINE LEARNING.pdf
PRABHUCECC
 
PDF
Artificial Neural Network
Atul Krishna
 
PPTX
Module1 (2).pptxvgybhunjimko,l.vgbyhnjmk;
vallepubalaji66
 
PPT
feedforward-network-
neelamsanjeevkumar
 
PPT
cs621-lect18-feedforward-network-contd-2009-9-24.ppt
GayathriRHICETCSESTA
 
PPT
cs621-lect18-feedforward-network-contd-2009-9-24.ppt
GayathriRHICETCSESTA
 
PPT
Neural networks,Single Layer Feed Forward
RohiniRajaramPandian
 
PPTX
Introduction to Neural Netwoks
Abdallah Bashir
 
PPT
introduction to neural networks ANN deep
RevathiSoundiran1
 
PPTX
Linear Algebra and Matlab tutorial
Jia-Bin Huang
 
PPT
SOFTCOMPUTERING TECHNICS - Unit
sravanthi computers
 
PPTX
Deep learning study 2
San Kim
 
PPT
nural network ER. Abhishek k. upadhyay
abhishek upadhyay
 
CS767_Lecture_04.pptx
ShujatHussainGadi
 
Neural Networks
Makerere Unversity School of Public Health, Victoria University
 
Artificial neural networks - A gentle introduction to ANNS.pptx
AttaNox1
 
Machine learning with neural networks
Let's talk about IT
 
ANN presentation explaination and architecture.pptx
Account1850
 
Artificial Neural Networks Lect2: Neurobiology & Architectures of ANNS
Mohammed Bennamoun
 
ARTIFICIAL-NEURAL-NETWORKMACHINELEARNING
mohanapriyastp
 
Chapter8 LINEAR DESCRIMINANT FOR MACHINE LEARNING.pdf
PRABHUCECC
 
Artificial Neural Network
Atul Krishna
 
Module1 (2).pptxvgybhunjimko,l.vgbyhnjmk;
vallepubalaji66
 
feedforward-network-
neelamsanjeevkumar
 
cs621-lect18-feedforward-network-contd-2009-9-24.ppt
GayathriRHICETCSESTA
 
cs621-lect18-feedforward-network-contd-2009-9-24.ppt
GayathriRHICETCSESTA
 
Neural networks,Single Layer Feed Forward
RohiniRajaramPandian
 
Introduction to Neural Netwoks
Abdallah Bashir
 
introduction to neural networks ANN deep
RevathiSoundiran1
 
Linear Algebra and Matlab tutorial
Jia-Bin Huang
 
SOFTCOMPUTERING TECHNICS - Unit
sravanthi computers
 
Deep learning study 2
San Kim
 
nural network ER. Abhishek k. upadhyay
abhishek upadhyay
 

More from Randa Elanwar (20)

PDF
الجزء السادس ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الخامس ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الرابع ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الثالث ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الثاني ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الأول ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (الترجمة والتلخيص )_Pdf5of5
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (القصة القصيرة والخاطرة والأخطاء ال...
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (مقالات الموارد )_Pdf3of5
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (المقالات الإخبارية )_Pdf2of5
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (المقالات المبنية على البحث )_Pdf1of5
Randa Elanwar
 
PDF
تعريف بمدونة علماء مصر ومحاور التدريب على الكتابة للمدونين
Randa Elanwar
 
PDF
Entrepreneurship_who_is_your_customer_(arabic)_7of7
Randa Elanwar
 
PDF
Entrepreneurship_who_is_your_customer_(arabic)_5of7
Randa Elanwar
 
PDF
Entrepreneurship_who_is_your_customer_(arabic)_4of7
Randa Elanwar
 
PDF
Entrepreneurship_who_is_your_customer_(arabic)_2of7
Randa Elanwar
 
PDF
يوميات طالب بدرجة مشرف (Part 19 of 20)
Randa Elanwar
 
PDF
يوميات طالب بدرجة مشرف (Part 18 of 20)
Randa Elanwar
 
PDF
يوميات طالب بدرجة مشرف (Part 17 of 20)
Randa Elanwar
 
PDF
يوميات طالب بدرجة مشرف (Part 16 of 20)
Randa Elanwar
 
الجزء السادس ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الخامس ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الرابع ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الثالث ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الثاني ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الأول ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (الترجمة والتلخيص )_Pdf5of5
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (القصة القصيرة والخاطرة والأخطاء ال...
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (مقالات الموارد )_Pdf3of5
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (المقالات الإخبارية )_Pdf2of5
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (المقالات المبنية على البحث )_Pdf1of5
Randa Elanwar
 
تعريف بمدونة علماء مصر ومحاور التدريب على الكتابة للمدونين
Randa Elanwar
 
Entrepreneurship_who_is_your_customer_(arabic)_7of7
Randa Elanwar
 
Entrepreneurship_who_is_your_customer_(arabic)_5of7
Randa Elanwar
 
Entrepreneurship_who_is_your_customer_(arabic)_4of7
Randa Elanwar
 
Entrepreneurship_who_is_your_customer_(arabic)_2of7
Randa Elanwar
 
يوميات طالب بدرجة مشرف (Part 19 of 20)
Randa Elanwar
 
يوميات طالب بدرجة مشرف (Part 18 of 20)
Randa Elanwar
 
يوميات طالب بدرجة مشرف (Part 17 of 20)
Randa Elanwar
 
يوميات طالب بدرجة مشرف (Part 16 of 20)
Randa Elanwar
 

Recently uploaded (20)

PDF
102 student loan defaulters named and shamed – Is someone you know on the list?
Kweku Zurek
 
PDF
Sports Quiz easy sports quiz sports quiz
nikunj2757
 
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
PDF
01-Introduction-to-Information-Management.pdf
PrinceIbarra
 
PDF
STATICS OF THE RIGID BODIES Hibbelers.pdf
pascualasturiaskaye4
 
PPTX
Renaissance Architecture: A Journey from Faith to Humanism
DrMonicaSharma
 
PDF
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
PDF
RMMM.pdf make it easy to upload and study
sajedul2
 
PDF
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
HakHe
 
PPTX
Final Presentation General Medicine 03-08-2024.pptx
ZaheerAhmad228692
 
PPTX
Cell Types and Its function , kingdom of life
ClaireMangundayao1
 
PDF
Basic Mud Logging Guide for educational purpose
wdandworks
 
PDF
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
MarjorieLopezTiu
 
PPTX
human mycosis Human fungal infections are called human mycosis..pptx
shilpa939953
 
PDF
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
PDF
Abdominal Access Techniques with Prof. Dr. R K Mishra
mohitsuren827
 
PDF
TR - Agricultural Crops Production NC III.pdf
avgilmegino3
 
PPTX
Pharmacology of Heart Failure /Pharmacotherapy of CHF
Rajshri Ghogare
 
PPTX
Microbial diseases, their pathogenesis and prophylaxis
DevlinaSengupta
 
PDF
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Academy of Grassroots Studies and Research of India (AGRASRI)
 
102 student loan defaulters named and shamed – Is someone you know on the list?
Kweku Zurek
 
Sports Quiz easy sports quiz sports quiz
nikunj2757
 
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
01-Introduction-to-Information-Management.pdf
PrinceIbarra
 
STATICS OF THE RIGID BODIES Hibbelers.pdf
pascualasturiaskaye4
 
Renaissance Architecture: A Journey from Faith to Humanism
DrMonicaSharma
 
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
RMMM.pdf make it easy to upload and study
sajedul2
 
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
HakHe
 
Final Presentation General Medicine 03-08-2024.pptx
ZaheerAhmad228692
 
Cell Types and Its function , kingdom of life
ClaireMangundayao1
 
Basic Mud Logging Guide for educational purpose
wdandworks
 
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
MarjorieLopezTiu
 
human mycosis Human fungal infections are called human mycosis..pptx
shilpa939953
 
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
Abdominal Access Techniques with Prof. Dr. R K Mishra
mohitsuren827
 
TR - Agricultural Crops Production NC III.pdf
avgilmegino3
 
Pharmacology of Heart Failure /Pharmacotherapy of CHF
Rajshri Ghogare
 
Microbial diseases, their pathogenesis and prophylaxis
DevlinaSengupta
 
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Academy of Grassroots Studies and Research of India (AGRASRI)
 

Introduction to Neural networks (under graduate course) Lecture 5 of 9

  • 1. Neural Networks Dr. Randa Elanwar Lecture 5
  • 2. Lecture Content • Linearly separable functions: 2-class problem implementation – Learning laws: Perceptron learning rule – Pattern mode solution method – Batch mode solution method 2Neural Networks Dr. Randa Elanwar
  • 3. Linearly Separable Functions • Example: logical OR, with initial weights 0.5, 0.3 with bias = 0.5 and activation step function at t=0.5. The learning rate = 1 3Neural Networks Dr. Randa Elanwar x2 w1= 0.5 w2 = 0.3 x1 yin = x1w1 + x2w2   y Activation Function: Binary Step Function t = 0.5, (y-in) = 1 if y-in >= t otherwise (y-in) = 0
  • 4. Solving Linearly Separable Functions (Pattern mode) • Given: • Since we consider bias as additional weight thus the weight vector is 1x3 we have to fix the dimensionality of the input vector x1, x2, x3 and x4 from 2x1 to be 3x1 to perform the multiplication. 4Neural Networks Dr. Randa Elanwar            11 10 01 00 X).( bXWfY  x1 x2 x3 x4 x1 x2 y 0 0 0 0 1 1 1 0 1 1 1 1           111 101 011 001 X  5.03.05.0)0( W
  • 5. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 1 5Neural Networks Dr. Randa Elanwar OK Wrong 1,1 1 0 1 ].5.03.15.0[3.)1(           yXW 1,3.2 1 1 1 ].5.03.15.0[4.)1(           yXW OK OK Wrong1,5.0 1 0 0 ].5.03.15.0[1.)1(           yXW   0,5.0 1 0 0 .5.03.05.01.)0(           yXW   0,2.0 1 1 0 .5.03.05.02.)0(           yXW          5.0 3.1 5.0 )..( 2)0()1( XyWW ydis TT 
  • 6. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 2 • Update weight vector for iteration 3 6Neural Networks Dr. Randa Elanwar           5.0 3.1 5.0 1)..()1()2( XyyWW dis TT  1,8.0 1 1 0 ].5.03.15.0[2.)2(           yXW 0,0 1 0 1 ].5.03.15.0[3.)2(           yXW OK Wrong          5.0 3.1 5.1 3)..()2()3( XyyWW dis TT  1,5.0 1 0 0 ].5.03.15.1[1.)3(           yXW 1,3.3 1 1 1 ].5.03.15.1[4.)3(           yXW OK Wrong
  • 7. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 4 • The weights learning has converged at 4 iterations 7Neural Networks Dr. Randa Elanwar 0,5.0 1 0 0 ].5.03.15.2[1.)4(           yXW 1,8.0 1 1 0 ].5.03.15.1[2.)4(           yXW           5.0 3.1 5.1 1)..()3()4( XyyWW dis TT  1,1 1 0 1 ].5.03.15.1[3.)4(           yXW 1,3.2 1 1 1 ].5.03.15.1[4.)4(           yXW OK OK OK OK
  • 8. Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 1 • Add w for all misclassified inputs together in 1 step 8Neural Networks Dr. Randa Elanwar 0,0 1 0 1 ].5.03.05.0[3.)0(           yXW 0,3.0 1 1 1 ].5.03.05.0[4.)0(           yXW OK Wrong Wrong Wrong 4)..(3)..(2)..()0()1( XyXyXy yyyWW disdisdis TT                                                 5.2 3.2 5.2 1 1 1 1 0 1 1 1 0 5.0 3.0 5.0 )1(W T   0,5.0 1 0 0 .5.03.05.01.)0(           yXW   0,2.0 1 1 0 .5.03.05.02.)0(           yXW
  • 9. Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 1 • Add w for all misclassified inputs together in 1 step 9Neural Networks Dr. Randa Elanwar 1,5.2 1 0 0 ].5.23.25.2[1.)1(           yXW 1,8.4 1 1 0 ].5.23.25.2[2.)1(           yXW 1,5 1 0 1 ].5.23.25.2[3.)1(           yXW 1,3.7 1 1 1 ].5.23.25.2[4.)1(           yXW Wrong OK OK OK          5.1 3.2 5.2 1)..()1()2( XyyWW dis TT 
  • 10. Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 3 • Add w for all misclassified inputs together in 1 step 10Neural Networks Dr. Randa Elanwar 1,5.1 1 0 0 ].5.13.25.2[1.)2(           yXW 1,8.3 1 1 0 ].5.13.25.2[2.)2(           yXW 1,4 1 0 1 ].5.13.25.2[3.)2(           yXW 1,3.6 1 1 1 ].5.13.25.2[4.)2(           yXW OK OK OK Wrong          5.0 3.2 5.2 1)..()2()3( XyyWW dis TT 
  • 11. Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 4 • Add w for all misclassified inputs together in 1 step 11Neural Networks Dr. Randa Elanwar OK OK OK Wrong           5.0 3.2 5.2 1)..()3()4( XyyWW dis TT  1,5.0 1 0 0 ].5.03.25.2[1.)3(           yXW 1,8.2 1 1 0 ].5.03.25.2[2.)3(           yXW 1,3 1 0 1 ].5.03.25.2[3.)3(           yXW 1,3.5 1 1 1 ].5.03.25.2[4.)3(           yXW
  • 12. Solving Linearly Separable Functions (Batch mode) • The weights learning has also converged at 4 iterations but with different final values 12Neural Networks Dr. Randa Elanwar OK OK OK OK0,5.0 1 0 0 ].5.03.25.2[1.)4(           yXW 1,8.1 1 1 0 ].5.03.25.2[2.)4(           yXW 1,2 1 0 1 ].5.03.25.2[3.)4(           yXW 1,3.4 1 1 1 ].5.03.25.2[4.)4(           yXW
  • 13. Linearly Separable Functions • Example: Consider linearly separable patterns where class C1 consists of the two patterns [1 0]T and [1 1]T and C2 consists of the two patterns [0 0]T and [0 1]T. Use the perceptron algorithm with  = 1 and w(0)= [0 1 -1/2]T to design the line separating the two classes. 13Neural Networks Dr. Randa Elanwar X2 X1 X4 X3 X2 X1 initial finalIt is very important to graph the problem to define the initial line and assign the direction of positive and negative Initial weight  X2-1/2 = 0, intersects vertical axis at (0,1/2) and is parallel to horizontal axis When X2>1/2 we get +ve value thus the positive direction is above the line +ve -ve
  • 14. Linearly Separable Functions • Initially x2 and x3 are in the correct side while x1 and x4 are in the wrong side. • Thus we assume x1 and x2 have to be on the +ve side and x3 and x4 have to be on the –ve side • Note that sometimes you are not given the activation function f. In such case: you can compute outputs and update weights by polarity (sign) instead of the activation function value: If the W.X >0 and it is wrong w = .(-1).X If the W.X <0 and it is wrong w = .(+1).X 14Neural Networks Dr. Randa Elanwar
  • 15. Solving Linearly Separable Functions (Pattern mode) • Again since the weight vector is 1x3 we have to fix the dimensionality of the input vector x1, x2, x3 and x4 from 2x1 to be 3x1 to perform the multiplication. • Update weight vector for iteration 1 15Neural Networks Dr. Randa Elanwar 5.0 1 0 1 ].5.010[1.)0(          XW -ve and it have to be +ve          5.0 1 1 1)0()0()1( Xw WWW TTT 5.2 1 1 1 ].5.011[2.)1(          XW OK 5.0 1 0 0 ].5.011[3.)1(          XW +ve and it have to be -ve
  • 16. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 2 • Update weight vector for iteration 3 • Update weight vector for iteration 4 16Neural Networks Dr. Randa Elanwar           5.0 1 1 3)1()1()2( Xw WWW TTT 5.0 1 1 0 ].5.011[4.)2(          XW +ve and it have to be -ve           5.1 0 1 4)2()2()3( Xw WWW TTT 5.0 1 0 1 ].5.101[1.)3(          XW -ve and it have to be +ve           5.0 0 2 1)3()3()4( Xw WWW TTT 5.1 1 1 1 ].5.002[2.)4(          XW OK
  • 17. Solving Linearly Separable Functions (Pattern mode) • The new straight line equation is 2X1-1/2 = 0 a vertical line intersecting the horizontal axis at 1/4 and is parallel to the vertical axis • The solution has converged in 4 iterations. 17Neural Networks Dr. Randa Elanwar   5.0 1 0 0 5.0023.)4(          XW OK   5.0 1 1 0 5.0024.)4(          XW OK   5.1 1 0 1 5.0021.)4(          XW OK
  • 18. Solving Linearly Separable Functions (Batch mode) • Again since the weight vector is 3x1 we have to fix the dimensionality of the input vector x1, x2, x3 and x4 to be 1x3 to perform the multiplication. • Update weight vector for iteration 1. Add w for all misclassified inputs together in 1 step 18Neural Networks Dr. Randa Elanwar +ve and it have to be -ve -ve and it have to be +ve OK OK 4.1.)0()1( XXWW TT                                         5.0 0 1 1 1 0 1 0 1 5.0 1 0 )1(W T 5.0 1 0 1 ].5.010[1.)0(          XW 1 1 1 1 ].5.010[2.)0(          XW 5.0 1 0 0 ].5.010[3.)0(          XW 5.0 1 1 0 ].5.010[4.)0(          XW
  • 19. Solving Linearly Separable Functions (Batch mode) • The new straight line equation is X1-1/2 = 0 a vertical line intersecting the horizontal axis at 1/2 and is parallel to the vertical axis • The solution has converged in 1 iteration. 19Neural Networks Dr. Randa Elanwar OK OK OK OK5.0 1 0 1 ].5.001[1.)1(          XW 5.0 1 1 1 ].5.001[2.)1(          XW 5.0 1 0 0 ].5.001[3.)1(          XW 5.0 1 1 0 ].5.001[4.)1(          XW
  • 20. Non linear problems • XOR problem • No way to draw a line to separate the positive from negative examples 20Neural Networks Dr. Randa Elanwar Input1 Input2 Output 0 0 0 0 1 1 1 0 1 1 1 0
  • 21. Non linear problems • XOR problem • The only way to separate the positive from negative examples is to draw 2 lines (i.e., we need 2 straight line equations) or nonlinear region to capture one type only 21Neural Networks Dr. Randa Elanwar +ve +ve -ve -ve+ve -ve cba yx  22
  • 22. Non linear problems • To implement the nonlinearity we need to insert one or more extra layer of nodes between the input layer and the output layer (Hidden layer) 22Neural Networks Dr. Randa Elanwar