SlideShare a Scribd company logo

Introduction to Neural networks (under graduate course) Lecture 4 of 9

Download as PPTX, PDF
Randa Elanwar, profile picture
Randa Elanwar

This document discusses the fundamentals of neural networks, focusing on training methods for linearly separable functions using the perceptron learning rule. It explains how weights are adjusted based on the error between desired and calculated outputs, detailing the iterative process involved in achieving convergence. Additionally, it introduces concepts such as node biases and the significance of learning rates in the training of neural networks.

Education
Related topics:
Neural Networks
1 of 22
Downloaded 80 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Neural Networks Dr. Randa Elanwar Lecture 4
Lecture Content • Linearly separable functions: logical gate implementation – Learning laws: Perceptron learning rule – Pattern mode solution method – Batch mode solution method 2Neural Networks Dr. Randa Elanwar
Learning Linearly Separable Functions • Initial network has a randomly assigned weights. • Learning is done by making small adjustments in the weights to reduce the difference between the observed and predicted values. • Main difference from the logical algorithms is the need to repeat the update phase several times in order to achieve convergence. • Updating process is divided into epochs. • Each epoch updates all the weights of the process. • Note that: the initial weights and the learning rate value determine the number of iterations needed for conversion. 3Neural Networks Dr. Randa Elanwar
Perceptron learning rule • Desired the desired output for a given input • Network calculates what it thinks the output should be • Network changes its weights in proportion to the error between the desired & calculated results • wi,j =  * [Desiredi - outputi] * inputj – where: –  is the learning rate (given constant); – Desiredi - outputi is the error term; – and inputj is the input activation • wi,j = wi,j + wi,j (delta rule) • Note: there are other learning rules/laws that will be discussed later • Learning rate : (1)Used to control the amount of weight adjustment at each step of training, (2) ranges from 0 to 1, (3) determines the rate of learning in each time step 4Neural Networks Dr. Randa Elanwar
Adjusting perceptron weights • wi,j = wi,j + wi,j • wi,j =  * [Desiredi - outputi] * inputj • missi is (Desiredi - outputi) • Adjust each wi,j based on inputj and missi • If a set of <input, output> pairs are learnable (representable), the delta rule will find the necessary weights (when miss=0) – in a finite number of steps – independent of initial weights Desired < 0, output > 0 w<0 Desired = 0, output = 0 w=0 Desired > 0, output < 0 w>0 5Neural Networks Dr. Randa Elanwar
Hypothetical example • Suppose we have 2 glasses: first is narrow and tall and has water in it, second is wide and short with no water in it • Target is to make both glasses contain the same volume of water • Initially, we add some water from the tall to the short then we measure volumes • If the volume in the short is less than the tall we add more water • If the volume in the short is more than the tall we return back some water • And so on till: If both volumes are equal we are done • The target = desired output, water = weights, difference measure = error 6Neural Networks Dr. Randa Elanwar
Node biases • A node’s output is a weighted function of its inputs • What is a bias? • How can we learn the bias value? • Answer: treat them like just another weight 7Neural Networks Dr. Randa Elanwar
Training biases () • A node’s output: – 1 if w1x1 + w2x2 + … + wnxn >=  – 0 otherwise • Rewrite – w1x1 + w2x2 + … + wnxn -  >= 0 – w1x1 + w2x2 + … + wnxn + (-1) >= 0 • Hence, the bias is just another weight whose activation is always -1 • Just add one more input unit to the network topology bias 8Neural Networks Dr. Randa Elanwar
Linearly Separable Functions • When solving the logical AND problem we are searching for the straight line equation separating +ve (1)and –ve (0) output regions on the graph • Different values for w1, w2, θ lead to different line slope. We have more than 1 solution depending on: initial weights W, learning rate , activation function f and learning mode (Pattern vs. Batch) 9Neural Networks Dr. Randa Elanwar  IwIw 2211 +ve +ve +ve +ve -ve -ve -ve -ve
Linearly Separable Functions • Similarly for the logical OR problem • Different values for w1, w2, θ lead to different line slope. • We have more than 1 solution depending on: initial weights W, learning rate , activation function f and learning mode (Pattern vs. Batch) 10Neural Networks Dr. Randa Elanwar  IwIw 2211 -ve -ve -ve -ve +ve +ve +ve +ve
Linearly Separable Functions • Example: logical AND, with initial weights 0.5, 0.3 with bias = 0.5 and activation step function at t=0.5. The learning rate = 1 11Neural Networks Dr. Randa Elanwar x2 w1= 0.5 w2 = 0.3 x1 yin = x1w1 + x2w2   y Activation Function: Binary Step Function t = 0.5, (y-in) = 1 if y-in >= t otherwise (y-in) = 0
Solving Linearly Separable Functions (Pattern mode) • Given: • Since we consider bias as additional weight thus the weight vector is 1x3 we have to fix the dimensionality of the input vector x1, x2, x3 and x4 from 2x1 to be 3x1 to perform the multiplication. 12Neural Networks Dr. Randa Elanwar  5.03.05.0)0( W            11 10 01 00 X).( bXWfY  x1 x2 x3 x4 x1 x2 y 0 0 0 0 1 0 1 0 0 1 1 1           111 101 011 001 X
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 1 13Neural Networks Dr. Randa Elanwar   0,5.0 1 0 0 .5.03.05.01.)0(           yXW OK OK OK Wrong          5.0 3.1 5.1 )..( 4)0()1( XyWW ydis TT    0,2.0 1 1 0 .5.03.05.02.)0(           yXW   0,0 1 0 1 .5.03.05.03.)0(           yXW   0,3.0 1 1 1 .5.03.05.04.)0(           yXW
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 2 • Update weight vector for iteration 3 14Neural Networks Dr. Randa Elanwar   1,5.0 1 0 0 .5.03.15.11.)1(           yXW Wrong           5.0 3.1 5.1 )..( 1)1()2( XyWW ydis TT    1,8.0 1 1 0 .5.03.15.12.)2(           yXW Wrong           5.1 3.0 5.1 )..( 2)2()3( XyWW ydis TT 
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 4 • Update weight vector for iteration 5 15Neural Networks Dr. Randa Elanwar   0,0 1 0 1 .5.13.05.13.)3(           yXW   0,3.0 1 1 1 .5.13.05.14.)3(           yXW OK Wrong   0,5.0 1 0 0 5.03.15.21.)4(           yXW OK Wrong  1,8.0 1 1 0 5.03.15.22.)4(           yXW           5.0 3.1 5.2 )..( 4)3()4( XyWW ydis TT            5.1 3.0 5.2 2)..()4()5( XyyWW dis TT 
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 6 • Update weight vector for iteration 7 16Neural Networks Dr. Randa Elanwar   1,1 1 0 1 .5.13.05.23.)5(           yXW Wrong           5.2 3.0 5.1 3)..()5()6( XyyWW dis TT    0,7.0 1 1 1 .5.23.05.14.)6(           yXW           5.1 3.1 5.2 4)..()6()7( XyyWW dis TT  Wrong   0,5.1 1 0 0 .5.13.15.21.)7(           yXW 0,2.0 1 1 0 .]5.13.15.2[2.)7(           yXW 1,1 1 0 1 ].5.13.15.2[3.)7(           yXW Wrong OK OK
Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 8 • Update weight vector for iteration 9 • Update weight vector for iteration 10 17Neural Networks Dr. Randa Elanwar           5.2 3.1 5.1 3)..()7()8( XyyWW dis TT  0,3.0 1 1 1 ].5.23.15.1[4.)8(           yXW Wrong           5.1 3.2 5.2 4)..()8()9( XyyWW dis TT  0,5.1 1 0 0 ].5.13.25.2[1.)9(           yXW 1,8.0 1 1 0 ].5.13.15.2[2.)9(           yXW Wrong OK           5.2 3.1 5.2 3)..()9()10( XyyWW dis TT 
Solving Linearly Separable Functions (Pattern mode) • The weights learning has converged at 10 iterations 18Neural Networks Dr. Randa Elanwar 0,0 1 0 1 ].5.13.15.2[3.)10(           yXW 1,3.1 1 1 1 ].5.23.15.2[4.)10(           yXW 0,5.2 1 0 0 ].5.13.15.2[1.)10(           yXW 0,7.0 1 1 0 ].5.13.15.2[2.)10(           yXW OK OK OK OK
Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 1 • Add w for all misclassified inputs together in 1 step 19Neural Networks Dr. Randa Elanwar 0,5.0 1 0 0 ].5.03.05.0[1.)0(           yXW 0,2.0 1 1 0 ].5.03.05.0[2.)0(           yXW 0,0 1 0 1 ].5.03.05.0[3.)0(           yXW 0,3.0 1 1 1 ].5.03.05.0[4.)0(           yXW OK OK OK Wrong          5.0 3.1 5.1 4)..()0()1( XyyWW dis TT 
Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 2 • Add w for all misclassified inputs together in 1 step 20Neural Networks Dr. Randa Elanwar 1,5.0 1 0 0 ].5.03.15.1[1.)1(           yXW 1,8.1 1 1 0 ].5.03.15.1[2.)1(           yXW 1,2 1 0 1 ].5.03.15.1[3.)1(           yXW 1,3.3 1 1 1 ].5.03.15.1[4.)1(           yXW Wrong Wrong Wrong OK 3)..(2)..(1)..()1()2( XyXyXy yyyWW disdisdis TT                                                 5.2 3.0 5.0 1 0 1 1 1 0 1 0 0 5.0 3.1 5.1 )2(W T
Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 3 • Add w for all misclassified inputs together in 1 step 21Neural Networks Dr. Randa Elanwar 0,5.2 1 0 0 ].5.23.05.0[1.)2(           yXW 0,2.2 1 1 0 ].5.23.05.0[2.)2(           yXW 0,2 1 0 1 ].5.23.05.0[3.)2(           yXW 0,7.1 1 1 1 ].5.23.05.0[4.)2(           yXW OK OK OK Wrong           5.1 3.1 5.1 4)..()2()3( XyyWW dis TT 
Solving Linearly Separable Functions (Batch mode) • Note that • The number of iterations in Batch mode solution is sometimes less than those of pattern mode • The final weights obtained by Batch mode solution are different from those obtained by pattern mode solution. 22Neural Networks Dr. Randa Elanwar 0,5.1 1 0 0 ].5.13.15.1[1.)3(           yXW 0,2.0 1 1 0 ].5.13.15.1[2.)3(           yXW 0,0 1 0 1 ].5.13.15.1[3.)3(           yXW 1,3.1 1 1 1 ].5.13.15.1[4.)3(           yXW OK OK OK OK

More Related Content

PPTX
Introduction to Neural networks (under graduate course) Lecture 5 of 9
Randa Elanwar
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 6 of 9
Randa Elanwar
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 2 of 9
Randa Elanwar
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 8 of 9
Randa Elanwar
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 7 of 9
Randa Elanwar
 
PPTX
03 Single layer Perception Classifier
Tamer Ahmed Farrag, PhD
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 9 of 9
Randa Elanwar
 
PPTX
Machine Learning - Neural Networks - Perceptron
Andrew Ferlitsch
 
Introduction to Neural networks (under graduate course) Lecture 5 of 9
Randa Elanwar
 
Introduction to Neural networks (under graduate course) Lecture 6 of 9
Randa Elanwar
 
Introduction to Neural networks (under graduate course) Lecture 2 of 9
Randa Elanwar
 
Introduction to Neural networks (under graduate course) Lecture 8 of 9
Randa Elanwar
 
Introduction to Neural networks (under graduate course) Lecture 7 of 9
Randa Elanwar
 
03 Single layer Perception Classifier
Tamer Ahmed Farrag, PhD
 
Introduction to Neural networks (under graduate course) Lecture 9 of 9
Randa Elanwar
 
Machine Learning - Neural Networks - Perceptron
Andrew Ferlitsch
 

What's hot (20)

PDF
Artificial neural networks
arjitkantgupta
 
PDF
Artificial Neural Networks Lect2: Neurobiology & Architectures of ANNS
Mohammed Bennamoun
 
PPTX
Unit 1
Vinod Srinivasan
 
PPT
lecture07.ppt
butest
 
PPTX
Fuzzy and nn
Shimi Haridasan
 
PPT
Multilayer perceptron
smitamm
 
PPTX
Neural Networks
Sagacious IT Solution
 
PPT
Artificial Neural Networks
Arslan Zulfiqar
 
PPTX
04 Multi-layer Feedforward Networks
Tamer Ahmed Farrag, PhD
 
PPSX
Perceptron (neural network)
EdutechLearners
 
PPTX
Basic Learning Algorithms of ANN
waseem khan
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 3 of 9
Randa Elanwar
 
PDF
Introduction to Artificial Neural Network
Qingkai Kong
 
PDF
Artificial Neural Network Lect4 : Single Layer Perceptron Classifiers
Mohammed Bennamoun
 
PDF
Introduction to Artificial Neural Networks
Stratio
 
PPTX
Artificial Neural Network
Dessy Amirudin
 
PPTX
Deep learning algorithms
Revanth Kumar
 
PPTX
Deep neural networks & computational graphs
Revanth Kumar
 
PPTX
Regression and Classification: An Artificial Neural Network Approach
Khulna University
 
ODP
Artificial Neural Network
Knoldus Inc.
 
Artificial neural networks
arjitkantgupta
 
Artificial Neural Networks Lect2: Neurobiology & Architectures of ANNS
Mohammed Bennamoun
 
Unit 1
Vinod Srinivasan
 
lecture07.ppt
butest
 
Fuzzy and nn
Shimi Haridasan
 
Multilayer perceptron
smitamm
 
Neural Networks
Sagacious IT Solution
 
Artificial Neural Networks
Arslan Zulfiqar
 
04 Multi-layer Feedforward Networks
Tamer Ahmed Farrag, PhD
 
Perceptron (neural network)
EdutechLearners
 
Basic Learning Algorithms of ANN
waseem khan
 
Introduction to Neural networks (under graduate course) Lecture 3 of 9
Randa Elanwar
 
Introduction to Artificial Neural Network
Qingkai Kong
 
Artificial Neural Network Lect4 : Single Layer Perceptron Classifiers
Mohammed Bennamoun
 
Introduction to Artificial Neural Networks
Stratio
 
Artificial Neural Network
Dessy Amirudin
 
Deep learning algorithms
Revanth Kumar
 
Deep neural networks & computational graphs
Revanth Kumar
 
Regression and Classification: An Artificial Neural Network Approach
Khulna University
 
Artificial Neural Network
Knoldus Inc.
 
Ad

Similar to Introduction to Neural networks (under graduate course) Lecture 4 of 9 (20)

PDF
ARTIFICIAL-NEURAL-NETWORKMACHINELEARNING
mohanapriyastp
 
PDF
NN-Ch2.PDF
gnans Kgnanshek
 
PDF
Machine Learning 1
cairo university
 
PPTX
latest TYPES OF NEURAL NETWORKS (2).pptx
MdMahfoozAlam5
 
PPT
ch11.pptKGYUTFYDRERLJIOUY7T867RVHOJIP09-IU08Y7GTFGYU890-I90UIYGUI
Abodahab
 
PPT
ch11.ppt kusrdsdagrfzgfdfgdfsdsfdsxgdhfjgh50s
Abodahab
 
PPT
Lec1 Inroduction to Neural Network.ppt
MohamedSadek942403
 
PPT
SOFTCOMPUTERING TECHNICS - Unit
sravanthi computers
 
PPTX
CS532L4_Backpropagation.pptx
MFaisalRiaz5
 
PPT
Artificial Neural Network
Pratik Aggarwal
 
PPTX
Comprehensive Guide to Neural Networks in Machine Learning and Deep Learning ...
RajeswariBsr1
 
PPTX
ACUMENS ON NEURAL NET AKG 20 7 23.pptx
gnans Kgnanshek
 
PPT
Function Approx2009
Imthias Ahamed
 
PDF
Setting Artificial Neural Networks parameters
Madhumita Tamhane
 
PDF
Lesson 38
Avijit Kumar
 
PDF
AI Lesson 38
Assistant Professor
 
PPTX
Artificial neural networks - A gentle introduction to ANNS.pptx
AttaNox1
 
PDF
lecture24NeuralNetworksNYUcourseEEEE.pdf
fankerui92
 
PPT
19_Learning.ppt
gnans Kgnanshek
 
PPTX
Neural Networks
Makerere Unversity School of Public Health, Victoria University
 
ARTIFICIAL-NEURAL-NETWORKMACHINELEARNING
mohanapriyastp
 
NN-Ch2.PDF
gnans Kgnanshek
 
Machine Learning 1
cairo university
 
latest TYPES OF NEURAL NETWORKS (2).pptx
MdMahfoozAlam5
 
ch11.pptKGYUTFYDRERLJIOUY7T867RVHOJIP09-IU08Y7GTFGYU890-I90UIYGUI
Abodahab
 
ch11.ppt kusrdsdagrfzgfdfgdfsdsfdsxgdhfjgh50s
Abodahab
 
Lec1 Inroduction to Neural Network.ppt
MohamedSadek942403
 
SOFTCOMPUTERING TECHNICS - Unit
sravanthi computers
 
CS532L4_Backpropagation.pptx
MFaisalRiaz5
 
Artificial Neural Network
Pratik Aggarwal
 
Comprehensive Guide to Neural Networks in Machine Learning and Deep Learning ...
RajeswariBsr1
 
ACUMENS ON NEURAL NET AKG 20 7 23.pptx
gnans Kgnanshek
 
Function Approx2009
Imthias Ahamed
 
Setting Artificial Neural Networks parameters
Madhumita Tamhane
 
Lesson 38
Avijit Kumar
 
AI Lesson 38
Assistant Professor
 
Artificial neural networks - A gentle introduction to ANNS.pptx
AttaNox1
 
lecture24NeuralNetworksNYUcourseEEEE.pdf
fankerui92
 
19_Learning.ppt
gnans Kgnanshek
 
Neural Networks
Makerere Unversity School of Public Health, Victoria University
 
Ad

More from Randa Elanwar (20)

PDF
الجزء السادس ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الخامس ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الرابع ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الثالث ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الثاني ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
الجزء الأول ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (الترجمة والتلخيص )_Pdf5of5
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (القصة القصيرة والخاطرة والأخطاء ال...
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (مقالات الموارد )_Pdf3of5
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (المقالات الإخبارية )_Pdf2of5
Randa Elanwar
 
PDF
تدريب مدونة علماء مصر على الكتابة الفنية (المقالات المبنية على البحث )_Pdf1of5
Randa Elanwar
 
PDF
تعريف بمدونة علماء مصر ومحاور التدريب على الكتابة للمدونين
Randa Elanwar
 
PDF
Entrepreneurship_who_is_your_customer_(arabic)_7of7
Randa Elanwar
 
PDF
Entrepreneurship_who_is_your_customer_(arabic)_5of7
Randa Elanwar
 
PDF
Entrepreneurship_who_is_your_customer_(arabic)_4of7
Randa Elanwar
 
PDF
Entrepreneurship_who_is_your_customer_(arabic)_2of7
Randa Elanwar
 
PDF
يوميات طالب بدرجة مشرف (Part 19 of 20)
Randa Elanwar
 
PDF
يوميات طالب بدرجة مشرف (Part 18 of 20)
Randa Elanwar
 
PDF
يوميات طالب بدرجة مشرف (Part 17 of 20)
Randa Elanwar
 
PDF
يوميات طالب بدرجة مشرف (Part 16 of 20)
Randa Elanwar
 
الجزء السادس ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الخامس ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الرابع ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الثالث ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الثاني ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
الجزء الأول ماذا ستقدم لعميلك ريادة الأعمال خطوة بخطوة
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (الترجمة والتلخيص )_Pdf5of5
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (القصة القصيرة والخاطرة والأخطاء ال...
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (مقالات الموارد )_Pdf3of5
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (المقالات الإخبارية )_Pdf2of5
Randa Elanwar
 
تدريب مدونة علماء مصر على الكتابة الفنية (المقالات المبنية على البحث )_Pdf1of5
Randa Elanwar
 
تعريف بمدونة علماء مصر ومحاور التدريب على الكتابة للمدونين
Randa Elanwar
 
Entrepreneurship_who_is_your_customer_(arabic)_7of7
Randa Elanwar
 
Entrepreneurship_who_is_your_customer_(arabic)_5of7
Randa Elanwar
 
Entrepreneurship_who_is_your_customer_(arabic)_4of7
Randa Elanwar
 
Entrepreneurship_who_is_your_customer_(arabic)_2of7
Randa Elanwar
 
يوميات طالب بدرجة مشرف (Part 19 of 20)
Randa Elanwar
 
يوميات طالب بدرجة مشرف (Part 18 of 20)
Randa Elanwar
 
يوميات طالب بدرجة مشرف (Part 17 of 20)
Randa Elanwar
 
يوميات طالب بدرجة مشرف (Part 16 of 20)
Randa Elanwar
 

Recently uploaded (20)

PDF
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
MarjorieLopezTiu
 
PPTX
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
NihumathunnisaH
 
PDF
Sports Quiz easy sports quiz sports quiz
nikunj2757
 
PPTX
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
chetansingh379583
 
PDF
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Academy of Grassroots Studies and Research of India (AGRASRI)
 
PPTX
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
kawisojames10
 
PPTX
Final Presentation General Medicine 03-08-2024.pptx
ZaheerAhmad228692
 
PDF
O7-L3 Supply Chain Operations - ICLT Program
labyankof
 
PDF
Anesthesia in Laparoscopic Surgery in India
mohitsuren827
 
PPTX
Institutional Correction lecture only . . .
limvtheghins
 
PDF
VCE English Exam - Section C Student Revision Booklet
jpinnuck
 
PDF
RMMM.pdf make it easy to upload and study
sajedul2
 
PDF
2.FourierTransform-ShortQuestionswithAnswers.pdf
Raji Murugan
 
PDF
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
PDF
102 student loan defaulters named and shamed – Is someone you know on the list?
Kweku Zurek
 
PDF
Basic Mud Logging Guide for educational purpose
wdandworks
 
PDF
Pre independence Education in Inndia.pdf
goofy5603
 
PDF
Microbial disease of the cardiovascular and lymphatic systems
KeyaSharma
 
PPTX
PPH.pptx obstetrics and gynecology in nursing
SrideviDevaraj5
 
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
MarjorieLopezTiu
 
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
NihumathunnisaH
 
Sports Quiz easy sports quiz sports quiz
nikunj2757
 
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
chetansingh379583
 
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Academy of Grassroots Studies and Research of India (AGRASRI)
 
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
kawisojames10
 
Final Presentation General Medicine 03-08-2024.pptx
ZaheerAhmad228692
 
O7-L3 Supply Chain Operations - ICLT Program
labyankof
 
Anesthesia in Laparoscopic Surgery in India
mohitsuren827
 
Institutional Correction lecture only . . .
limvtheghins
 
VCE English Exam - Section C Student Revision Booklet
jpinnuck
 
RMMM.pdf make it easy to upload and study
sajedul2
 
2.FourierTransform-ShortQuestionswithAnswers.pdf
Raji Murugan
 
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
102 student loan defaulters named and shamed – Is someone you know on the list?
Kweku Zurek
 
Basic Mud Logging Guide for educational purpose
wdandworks
 
Pre independence Education in Inndia.pdf
goofy5603
 
Microbial disease of the cardiovascular and lymphatic systems
KeyaSharma
 
PPH.pptx obstetrics and gynecology in nursing
SrideviDevaraj5
 
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 

Introduction to Neural networks (under graduate course) Lecture 4 of 9

  • 1. Neural Networks Dr. Randa Elanwar Lecture 4
  • 2. Lecture Content • Linearly separable functions: logical gate implementation – Learning laws: Perceptron learning rule – Pattern mode solution method – Batch mode solution method 2Neural Networks Dr. Randa Elanwar
  • 3. Learning Linearly Separable Functions • Initial network has a randomly assigned weights. • Learning is done by making small adjustments in the weights to reduce the difference between the observed and predicted values. • Main difference from the logical algorithms is the need to repeat the update phase several times in order to achieve convergence. • Updating process is divided into epochs. • Each epoch updates all the weights of the process. • Note that: the initial weights and the learning rate value determine the number of iterations needed for conversion. 3Neural Networks Dr. Randa Elanwar
  • 4. Perceptron learning rule • Desired the desired output for a given input • Network calculates what it thinks the output should be • Network changes its weights in proportion to the error between the desired & calculated results • wi,j =  * [Desiredi - outputi] * inputj – where: –  is the learning rate (given constant); – Desiredi - outputi is the error term; – and inputj is the input activation • wi,j = wi,j + wi,j (delta rule) • Note: there are other learning rules/laws that will be discussed later • Learning rate : (1)Used to control the amount of weight adjustment at each step of training, (2) ranges from 0 to 1, (3) determines the rate of learning in each time step 4Neural Networks Dr. Randa Elanwar
  • 5. Adjusting perceptron weights • wi,j = wi,j + wi,j • wi,j =  * [Desiredi - outputi] * inputj • missi is (Desiredi - outputi) • Adjust each wi,j based on inputj and missi • If a set of <input, output> pairs are learnable (representable), the delta rule will find the necessary weights (when miss=0) – in a finite number of steps – independent of initial weights Desired < 0, output > 0 w<0 Desired = 0, output = 0 w=0 Desired > 0, output < 0 w>0 5Neural Networks Dr. Randa Elanwar
  • 6. Hypothetical example • Suppose we have 2 glasses: first is narrow and tall and has water in it, second is wide and short with no water in it • Target is to make both glasses contain the same volume of water • Initially, we add some water from the tall to the short then we measure volumes • If the volume in the short is less than the tall we add more water • If the volume in the short is more than the tall we return back some water • And so on till: If both volumes are equal we are done • The target = desired output, water = weights, difference measure = error 6Neural Networks Dr. Randa Elanwar
  • 7. Node biases • A node’s output is a weighted function of its inputs • What is a bias? • How can we learn the bias value? • Answer: treat them like just another weight 7Neural Networks Dr. Randa Elanwar
  • 8. Training biases () • A node’s output: – 1 if w1x1 + w2x2 + … + wnxn >=  – 0 otherwise • Rewrite – w1x1 + w2x2 + … + wnxn -  >= 0 – w1x1 + w2x2 + … + wnxn + (-1) >= 0 • Hence, the bias is just another weight whose activation is always -1 • Just add one more input unit to the network topology bias 8Neural Networks Dr. Randa Elanwar
  • 9. Linearly Separable Functions • When solving the logical AND problem we are searching for the straight line equation separating +ve (1)and –ve (0) output regions on the graph • Different values for w1, w2, θ lead to different line slope. We have more than 1 solution depending on: initial weights W, learning rate , activation function f and learning mode (Pattern vs. Batch) 9Neural Networks Dr. Randa Elanwar  IwIw 2211 +ve +ve +ve +ve -ve -ve -ve -ve
  • 10. Linearly Separable Functions • Similarly for the logical OR problem • Different values for w1, w2, θ lead to different line slope. • We have more than 1 solution depending on: initial weights W, learning rate , activation function f and learning mode (Pattern vs. Batch) 10Neural Networks Dr. Randa Elanwar  IwIw 2211 -ve -ve -ve -ve +ve +ve +ve +ve
  • 11. Linearly Separable Functions • Example: logical AND, with initial weights 0.5, 0.3 with bias = 0.5 and activation step function at t=0.5. The learning rate = 1 11Neural Networks Dr. Randa Elanwar x2 w1= 0.5 w2 = 0.3 x1 yin = x1w1 + x2w2   y Activation Function: Binary Step Function t = 0.5, (y-in) = 1 if y-in >= t otherwise (y-in) = 0
  • 12. Solving Linearly Separable Functions (Pattern mode) • Given: • Since we consider bias as additional weight thus the weight vector is 1x3 we have to fix the dimensionality of the input vector x1, x2, x3 and x4 from 2x1 to be 3x1 to perform the multiplication. 12Neural Networks Dr. Randa Elanwar  5.03.05.0)0( W            11 10 01 00 X).( bXWfY  x1 x2 x3 x4 x1 x2 y 0 0 0 0 1 0 1 0 0 1 1 1           111 101 011 001 X
  • 13. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 1 13Neural Networks Dr. Randa Elanwar   0,5.0 1 0 0 .5.03.05.01.)0(           yXW OK OK OK Wrong          5.0 3.1 5.1 )..( 4)0()1( XyWW ydis TT    0,2.0 1 1 0 .5.03.05.02.)0(           yXW   0,0 1 0 1 .5.03.05.03.)0(           yXW   0,3.0 1 1 1 .5.03.05.04.)0(           yXW
  • 14. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 2 • Update weight vector for iteration 3 14Neural Networks Dr. Randa Elanwar   1,5.0 1 0 0 .5.03.15.11.)1(           yXW Wrong           5.0 3.1 5.1 )..( 1)1()2( XyWW ydis TT    1,8.0 1 1 0 .5.03.15.12.)2(           yXW Wrong           5.1 3.0 5.1 )..( 2)2()3( XyWW ydis TT 
  • 15. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 4 • Update weight vector for iteration 5 15Neural Networks Dr. Randa Elanwar   0,0 1 0 1 .5.13.05.13.)3(           yXW   0,3.0 1 1 1 .5.13.05.14.)3(           yXW OK Wrong   0,5.0 1 0 0 5.03.15.21.)4(           yXW OK Wrong  1,8.0 1 1 0 5.03.15.22.)4(           yXW           5.0 3.1 5.2 )..( 4)3()4( XyWW ydis TT            5.1 3.0 5.2 2)..()4()5( XyyWW dis TT 
  • 16. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 6 • Update weight vector for iteration 7 16Neural Networks Dr. Randa Elanwar   1,1 1 0 1 .5.13.05.23.)5(           yXW Wrong           5.2 3.0 5.1 3)..()5()6( XyyWW dis TT    0,7.0 1 1 1 .5.23.05.14.)6(           yXW           5.1 3.1 5.2 4)..()6()7( XyyWW dis TT  Wrong   0,5.1 1 0 0 .5.13.15.21.)7(           yXW 0,2.0 1 1 0 .]5.13.15.2[2.)7(           yXW 1,1 1 0 1 ].5.13.15.2[3.)7(           yXW Wrong OK OK
  • 17. Solving Linearly Separable Functions (Pattern mode) • Update weight vector for iteration 8 • Update weight vector for iteration 9 • Update weight vector for iteration 10 17Neural Networks Dr. Randa Elanwar           5.2 3.1 5.1 3)..()7()8( XyyWW dis TT  0,3.0 1 1 1 ].5.23.15.1[4.)8(           yXW Wrong           5.1 3.2 5.2 4)..()8()9( XyyWW dis TT  0,5.1 1 0 0 ].5.13.25.2[1.)9(           yXW 1,8.0 1 1 0 ].5.13.15.2[2.)9(           yXW Wrong OK           5.2 3.1 5.2 3)..()9()10( XyyWW dis TT 
  • 18. Solving Linearly Separable Functions (Pattern mode) • The weights learning has converged at 10 iterations 18Neural Networks Dr. Randa Elanwar 0,0 1 0 1 ].5.13.15.2[3.)10(           yXW 1,3.1 1 1 1 ].5.23.15.2[4.)10(           yXW 0,5.2 1 0 0 ].5.13.15.2[1.)10(           yXW 0,7.0 1 1 0 ].5.13.15.2[2.)10(           yXW OK OK OK OK
  • 19. Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 1 • Add w for all misclassified inputs together in 1 step 19Neural Networks Dr. Randa Elanwar 0,5.0 1 0 0 ].5.03.05.0[1.)0(           yXW 0,2.0 1 1 0 ].5.03.05.0[2.)0(           yXW 0,0 1 0 1 ].5.03.05.0[3.)0(           yXW 0,3.0 1 1 1 ].5.03.05.0[4.)0(           yXW OK OK OK Wrong          5.0 3.1 5.1 4)..()0()1( XyyWW dis TT 
  • 20. Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 2 • Add w for all misclassified inputs together in 1 step 20Neural Networks Dr. Randa Elanwar 1,5.0 1 0 0 ].5.03.15.1[1.)1(           yXW 1,8.1 1 1 0 ].5.03.15.1[2.)1(           yXW 1,2 1 0 1 ].5.03.15.1[3.)1(           yXW 1,3.3 1 1 1 ].5.03.15.1[4.)1(           yXW Wrong Wrong Wrong OK 3)..(2)..(1)..()1()2( XyXyXy yyyWW disdisdis TT                                                 5.2 3.0 5.0 1 0 1 1 1 0 1 0 0 5.0 3.1 5.1 )2(W T
  • 21. Solving Linearly Separable Functions (Batch mode) • Update weight vector for iteration 3 • Add w for all misclassified inputs together in 1 step 21Neural Networks Dr. Randa Elanwar 0,5.2 1 0 0 ].5.23.05.0[1.)2(           yXW 0,2.2 1 1 0 ].5.23.05.0[2.)2(           yXW 0,2 1 0 1 ].5.23.05.0[3.)2(           yXW 0,7.1 1 1 1 ].5.23.05.0[4.)2(           yXW OK OK OK Wrong           5.1 3.1 5.1 4)..()2()3( XyyWW dis TT 
  • 22. Solving Linearly Separable Functions (Batch mode) • Note that • The number of iterations in Batch mode solution is sometimes less than those of pattern mode • The final weights obtained by Batch mode solution are different from those obtained by pattern mode solution. 22Neural Networks Dr. Randa Elanwar 0,5.1 1 0 0 ].5.13.15.1[1.)3(           yXW 0,2.0 1 1 0 ].5.13.15.1[2.)3(           yXW 0,0 1 0 1 ].5.13.15.1[3.)3(           yXW 1,3.1 1 1 1 ].5.13.15.1[4.)3(           yXW OK OK OK OK