Jst part2
0 likes369 views
This document discusses neural network architectures and activation functions. It describes single and multiple layer fully connected networks as well as recurrent networks with and without hidden units. It also summarizes several common activation functions including hardlimiter, threshold, sigmoid, and hyperbolic tangent functions. It provides examples of representing Boolean functions using linear thresholds and perceptron training algorithms.
![6/10/20136/10/2013
• Representation of Boolean function AND
• Linear threshold is used
Perceptron Training
t = 0.0t = 0.0
YY
XX
WW11 = 1.5= 1.5
WW33 = 1= 1
--11
WW22 = 1= 1
11 ifif ΣΣ wwiixxii >t>t
OutputOutput== {{00 otherwiseotherwise
Perceptron Training
• Epoch
– Presentation of the entire training set to the neural network.
– In the case of the AND function an epoch consists of four sets
of inputs being presented to the network (i.e. [0,0], [0,1], [1,0],
[1,1])
• Error
– a simple definition of error
– The error value is the amount by which the value output by
the network differs from the target value.
– For example, if we required the network to output 0 and it
output a 1, then Error = -1
Sum of squaredSum of squared
errors :errors :](https://image.slidesharecdn.com/jstpart2-130824063507-phpapp01/85/Jst-part2-4-320.jpg)
![66//1010//20132013
Perceptron Training
• Target Value (T)
– Value required to be produced
– If we present the network with [1,1] for the AND function,
T will be 1
• Output (O)
– The output value from the neuron
• Ij - Inputs being presented to the neuron
• Wj - Weight from input neuron (Ij) to the output neuron
• LR( ) - The learning rate
This dictates how quickly the network converges
It is set by a matter of experimentation
η
Perceptron Training
• Algorithm
While epoch produces a non null errorWhile epoch produces a non null error
End WhileEnd While](https://image.slidesharecdn.com/jstpart2-130824063507-phpapp01/85/Jst-part2-5-320.jpg)
Jst part2
- 1. 66//1010//20132013 Fungsi Aktivasi • Fungsi aktivasi dengan notasi: µ(.) mendefinisikan nilai output dari suatu neuron dalam level aktivasi tertentu berdasarkan nilai output pengkombinasi linier ui. • Beberapa fungsi aktivasi yg biasa digunakan: – Hardlimiter – Threshold – Sigmoid – Tangen Hiperbolik Fungsi Aktivasi 1. Hardlimiter 2. Piecewise Linear
- 2. 66//1010//20132013 Fungsi Aktivasi 3. Threshold xx f(.)f(.) ++11 tt f(x) = 0 jika x ≤ t f(x) = 1 jika x > t Fungsi Aktivasi 4. Sigmoid 5. Tangen Hiperbolik
- 3. 6/10/20136/10/2013 Arsitektur JST Single layerSingle layer Multiple layerMultiple layer fully connectedfully connected Recurrent networkRecurrent network without hidden unitswithout hidden units inputsinputs outputsoutputs { } Recurrent networkRecurrent network with hidden unitswith hidden units Unit delayUnit delay operatoroperator Standard Activation Functions • The hard-limiting threshold function – Corresponds to the biological paradigm • either fires or not • Sigmoid functions ('S'-shaped curves) – The logistic function – The hyperbolic tangent (symmetrical) – Both functions have a simple differential – Only the shape is important )exp(1 1 )( av vf −+ =
- 4. 6/10/20136/10/2013 • Representation of Boolean function AND • Linear threshold is used Perceptron Training t = 0.0t = 0.0 YY XX WW11 = 1.5= 1.5 WW33 = 1= 1 --11 WW22 = 1= 1 11 ifif ΣΣ wwiixxii >t>t OutputOutput== {{00 otherwiseotherwise Perceptron Training • Epoch – Presentation of the entire training set to the neural network. – In the case of the AND function an epoch consists of four sets of inputs being presented to the network (i.e. [0,0], [0,1], [1,0], [1,1]) • Error – a simple definition of error – The error value is the amount by which the value output by the network differs from the target value. – For example, if we required the network to output 0 and it output a 1, then Error = -1 Sum of squaredSum of squared errors :errors :
- 5. 66//1010//20132013 Perceptron Training • Target Value (T) – Value required to be produced – If we present the network with [1,1] for the AND function, T will be 1 • Output (O) – The output value from the neuron • Ij - Inputs being presented to the neuron • Wj - Weight from input neuron (Ij) to the output neuron • LR( ) - The learning rate This dictates how quickly the network converges It is set by a matter of experimentation η Perceptron Training • Algorithm While epoch produces a non null errorWhile epoch produces a non null error End WhileEnd While