AILABS - Lecture Series - Is AI the New Electricity? Topic:- Classification and Estimation using Artificial Neural Network, Presented by - Prof. Dinabandhu Bhandari
0 likes123 views
1. The document discusses classification and estimation using artificial neural networks. It provides examples of classification problems from industries like mining and banking loan approval. 2. It describes the basic components of an artificial neural network including the feedforward architecture with multiple layers of neurons and the backpropagation algorithm for learning network weights. 3. Examples are given to illustrate how neural networks can perform nonlinear classification and estimation through combinations of linear perceptron units in multiple layers with the backpropagation algorithm for training the network weights.
![Daily Roadway Development for Mining
Serial
No
No of
Agitator
Trucks
[1 to 4]
Agitator
Capacity m3)
[5 to 10 m3]
No of
Kibble
Trucks
[1 to 5]
Kibble
Capacity m3)
[2 to 5 m3]
Distance to
Site (m)
[1 to 4 km]
Daily Shotcrete
Roadway
Development
(m3)
1 2 5 2 5 1000 49.93
2 2 5 2 5 1500 49.93
8 1 5 2 2.5 1500 24.98
9 1 5 2 2.5 2000 24.98
10 1 5 2 2.5 2500 24.98
79 2 5 1 4 1000 27.95
80 2 5 1 4 1500 23.95
81 2 5 1 4 2000 23.31
82 2 5 1 4 2500 19.96
83 2 5 1 4 3000 19.31
89 1 5 1 4 3000 11.98
90 1 5 1 4 3500 11.98](https://image.slidesharecdn.com/artificialneuralnetworkedited-180706090532/85/AILABS-Lecture-Series-Is-AI-the-New-Electricity-Topic-Classification-and-Estimation-using-Artificial-Neural-Network-Presented-by-Prof-Dinabandhu-Bhandari-6-320.jpg)
![Convergence of Backprop
Gradient descent converges to some local minimum perhaps
not global minimum
Add momentum term: wki(n)
wki(n) = dk(n) xi (n) + wki(n-1)
with [0,1]
Stochastic gradient descent
Train multiple nets with different initial weights](https://image.slidesharecdn.com/artificialneuralnetworkedited-180706090532/85/AILABS-Lecture-Series-Is-AI-the-New-Electricity-Topic-Classification-and-Estimation-using-Artificial-Neural-Network-Presented-by-Prof-Dinabandhu-Bhandari-28-320.jpg)
AILABS - Lecture Series - Is AI the New Electricity? Topic:- Classification and Estimation using Artificial Neural Network, Presented by - Prof. Dinabandhu Bhandari
- 1. Use and distribution limited solely to authorized personnel. (c) Copyright 2018 Lecture Series: AI is the New Electricity? AILABS Academy J-3, GP Block, Sector V, Salt Lake City, Kolkata, West Bengal 700091 Classification and Estimation using Artificial Neural Network Presented at AILABS Academy, Kolkata on April 18th 2018 Dinabandhu Bhandari
- 2. Contents Examples of Classification and Estimation Components of Learning ANN – Architecture and Back Propagation Classification and Estimation Some Industrial Problems Observation Discussion and Q&A
- 4. An article without any style of the paragraphs
- 5. The article after applying appropriate style
- 6. Daily Roadway Development for Mining Serial No No of Agitator Trucks [1 to 4] Agitator Capacity m3) [5 to 10 m3] No of Kibble Trucks [1 to 5] Kibble Capacity m3) [2 to 5 m3] Distance to Site (m) [1 to 4 km] Daily Shotcrete Roadway Development (m3) 1 2 5 2 5 1000 49.93 2 2 5 2 5 1500 49.93 8 1 5 2 2.5 1500 24.98 9 1 5 2 2.5 2000 24.98 10 1 5 2 2.5 2500 24.98 79 2 5 1 4 1000 27.95 80 2 5 1 4 1500 23.95 81 2 5 1 4 2000 23.31 82 2 5 1 4 2500 19.96 83 2 5 1 4 3000 19.31 89 1 5 1 4 3000 11.98 90 1 5 1 4 3500 11.98
- 7. Bank loan approval Feature Value Age 35 years Gender Male Debt 10,00,000 Salary 15,00,000 Years in home 5 years … … 1. Approve or not? 2. How Much?
- 9. Formulation – Classification System
- 10. Formulation – Learning (Classification) System
- 12. The linear model - Perceptron
- 13. The linear model - Perceptron
- 16. Artificial Neural Network (Feed-forward network)? A set of neurons (simple processing unit) arranged in number of layers. The neurons in each layer feed their output forward to the next layer until we get the final output from the neural network. There can be any number of hidden layers within a Feedforward network. The number of neurons can be completely arbitrary. 16
- 17. Neuron – a simple processing unit A neuron can have any number of inputs The inputs may be represented as x1, x2, x3… xn. And the corresponding weights for the inputs as w1, w2, w3… wn. Output a = x1w1+x2w2+x3w3... +xnwn 17
- 18. • No connections within a layer • No direct connections between input and output layers • Fully connected between layers • Number of output units need not equal number of input units • Number of hidden units per layer can be more or less than input or output units y f w x bi ij j i j m = + = ( ) 1 Often include bias as an extra weight 18 Properties of architecture
- 19. Perceptron Learning Theorem Recap: A perceptron (threshold unit) can learn anything that it can represent (i.e. anything separable with a hyperplane) 19
- 20. The Exclusive OR problem A Perceptron cannot represent Exclusive OR since it is not linearly separable. 20
- 21. 21
- 22. Minsky & Papert (1969) offered solution to XOR problem by combining perceptron unit responses using a second layer of Units. Piecewise linear classification using an MLP with threshold (perceptron) units 1 2 +1 +1 3 22
- 23. What do each of the layers do? 1st layer draws linear boundaries 2nd layer combines the boundaries 3rd layer can generate arbitrarily complex boundaries 23
- 24. 2. Backward pass phase: computes ‘error signal’, propagates the error backwards through network starting at output units (where the error is the difference between actual and desired output values) 1. Forward pass phase: computes ‘functional signal’, feed forward propagation of input pattern signals through network Two phases of Backpropagation learning algorithm ‘BP’ 24 Frank Rosenblatt in 1961
- 25. Backpropagation xk xi wki wjk dj dk yj Backward step: propagate errors from output to hidden layer Forward step: Propagate activation from input to output layer
- 26. Backpropagation Algorithm Initialize each wi to some small random value Until the termination condition is met, Do For each training example <(x1,…xn),t> Do Input the instance (x1,…,xn) to the network and compute the network outputs yk For each output unit k dk=yk(1-yk)(tk-yk) For each hidden unit h dh=yh(1-yh) k wh,k dk For each network weight wi,j Do wi,j=wi,j+wi,j where wi,j= dj xi,j
- 27. Backpropagation Gradient descent over entire network weight vector Easily generalized to arbitrary directed graphs Finds a local, not necessarily global error minimum -in practice often works well (can be invoked multiple times with different initial weights) Often include weight momentum term wi,j(n)= dj xi,j + wi,j (n-1) Minimizes error for training examples Will it generalize well to unseen instances (out of sample error; over-fitting)? Training can be slow typical 1000-10000 iterations
- 28. Convergence of Backprop Gradient descent converges to some local minimum perhaps not global minimum Add momentum term: wki(n) wki(n) = dk(n) xi (n) + wki(n-1) with [0,1] Stochastic gradient descent Train multiple nets with different initial weights
- 29. Ways to use weight derivatives How often to update after each training case? after a full sweep through the training data? How much to update Use a fixed learning rate? Adapt the learning rate? Add momentum?
- 30. Training This was a single iteration of back-prop Training requires many iterations with many training examples or epochs (one epoch is entire presentation of complete training set) It can be slow ! Parallel computation implementation is also possible 30
- 31. Training and testing data How many examples ? The more the merrier ! Disjoint training and testing data sets learn from training data but evaluate performance (generalization ability) on unseen test data Aim: minimize error on test data 31 Hoeffding’s Inequality In-sample error Out-of- sample error |H|, cardinility of hypothesis set Number of training samples
- 32. Estimation xk xi wki wjk dj dk yj Difference with Classifier: Only in the calculation of error
- 33. References
- 34. Thank You