Artificial_Neural_Network.pdf
- 2. Dr. TANUJA PATGAR Dept. of Electronics and Communication Dr. Ambedkar Institute of Technology Bangalore tanujapatgar.ec@drait.edu.in 2
- 3. Learning Objectives Fundamentals of ANN Architecture Mathematical Model Machine Learning Deep Learning
- 4. 4 Brain and Machine • The brain performs Pattern Recognition,Association, more Complex processing • The machine can perform Calculation,Precision and Logical operation
- 5. 5 Contrast in Architecture • The Von Neumann architecture uses a single processing unit with tens of millions of operations per second, Absolute arithmetic precision, A computer must be programmed explicitly • The brain uses many slow unreliable processors acting in parallel • The brain can learn by experiencing the world
- 6. 6 • Artificial Neural Network is a network of neurons with an information processing model. It is inspired by the biological nervous systems such as brain process information. • Neural network is either a biological network made up of real biological neurons.
- 7. ARTIFICIAL NEURON MODEL Inputs to the network are represented by the mathematical symbol, xn Each inputs are multiplied by a connection weight , wn Sum = w1 x1 + ……+ wnxn Products are simply summed, fed through the transfer function, f(n) to generate a result. f w1 w2 xn x2 x1 wn f(w1 x1 + ……+ wnxn)
- 8. Network Design Terminologies of ANN • How many nodes? • Determines number of network weights • How many layers? • How many nodes per layer? • Input Layer , Hidden Layer, Output Layer • Weights • Bias • Threshold • Learning rate
- 9. Biological to Artificial Neurons Hidden Nodes Output Nodes Input Nodes I1 I2 I3 I4 O1 O2 “Distributed processing and representation” 3-Layer Network has 2 active layers
- 10. Neural Network Modelling 3-4-4-4-1 5-2-5 10
- 11. Topologies of Neural Networks Fully connected feedforward recurrent (feedback connections)
- 12. NEURAL ARCHITECTURES Every node is connected to every other nodes, and connections are either excitatory (positive weights), inhibitory (negative weights), or irrelevant (almost zero weights). Networks in which nodes are partitioned into subsets called layers, with no connections from layer j to k if j > k. Input node Input node output node output node Hidden node Layer 1 Layer2 (Input layer) (Output layer) Hidden Layer fully connected network layered network
- 13. It is the subclass of layered networks in which there is no intra-layer connections. Connection may exist between any node in layer i and any node in layer j for i < j, but a connection is not allowed for i=j. Feedforward network Is a subclass of acyclic networks in which a connection is allowed from a node in layer i only to nodes in layer i+1 Layer 1 Layer2 Layer0 Layer3 (Input layer) (Output layer) Hidden Layer Layer 1 Layer2 Layer0 Layer3 (Input layer) (Output layer) Hidden Layer Acyclic network
- 14. • Neural networks architecture consists of several modules, with sparse interconnections between them. • Modules are organized in several different ways as Hierarchial organization, Successive, refinement, Input modularity Modular Neural Network
- 16. SINGLE AND MULTI-LAYERED NEURAL NETWORKS
- 17. Mathematical Neural Modelling x1 x2 X1 X2 w1 w2 Y y n X 1 1 2 2 in y x w x w = + ( ) in y f y =
- 18. Example: The AND Perceptron Use the step function for activation. Boolean value false is represented as number 0. Boolean value true is represented as number 1. 18 𝑧 = ℎ 𝒘𝑇𝒙 𝑥1 𝑥0 = 1 𝑥2 Output: 𝑧 false AND false = false false AND true = false true AND false = false true AND true = true
- 19. Neural Learning It is a process by which NN adapts itself to stimulus by making proper parameter adjustments, resulting in the production of desired response Two kinds of learning Parameter learning:- connection weights are updated Structure Learning:- change in network structure
- 20. Model Training The process of modifying the weights in connections between network layers with the objective of achieving expected output is called training a network. This is achieved through Supervised learning Unsupervised learning Reinforcement learning
- 21. Machine Learning Data and output is run on the computer to create a program. This program can be used in traditional programming. Machine learning is like Farming Gardening. Seeds are algorithms, Nutrients are data, user as Gardner and Plants are programs. 21
- 22. ML Algorithm Components Representation: How to represent knowledge. Examples include decision trees, sets of rules, instances, graphical models, neural networks, support vector machines, model ensembles and others. Evaluation: Way to evaluate candidate programs (hypotheses). Examples include accuracy, prediction and recall, squared error, likelihood, posterior probability, cost, margin, entropy k-L divergence and others. Optimization: Candidate programs are generated known as the search process. For example combinatorial optimization, convex optimization, constrained optimization 22
- 23. Classification of learning 1. Supervised Learning 2. Unsupervised Learning 23
- 24. Supervised learning Train the machine using data which is well labeled that means some data is already tagged with the correct answer. It indicates the presence of supervisor as a teacher. Basically supervised learning is a learning in which we teach. The machine is provided with new set of examples(data) so that supervised learning algorithm analyses the training data(set of training examples) and produces correct outcome from labeled data. 24
- 25. 25 For instance, suppose you are given a basket filled with different of fruits. Now the first step is to train the machine with all different fruits one by one like this:
- 26. • If shape of object is round and depression at top having color Red then it will be labeled as –Apple • If shape of object is long curving cylinder having color Green- Yellow then it will be labeled as –Banana • Now suppose after training the data, you have given a new separate fruit say Banana from basket and asked to identify it. • Since the machine has already learned things from previous data and this time have to use it wisely. It will first classify the fruit with its shape and color and would confirm the fruit name as BANANA and put it in Banana category. • Thus the machine learns the things from training data(basket containing fruits) and then apply the knowledge to test data(new fruit). 26
- 28. Unsupervised Learning • Machine using information that is neither classified nor labeled and allowing the algorithm to act on that information without guidance. • The task of machine is to group unsorted information according to similarities, patterns and differences without any prior training of data. • No teacher is provided that means no training will be given to the machine and is restricted to find hidden structure in unlabeled data by itself. . 28
- 29. 29 •The machine has no idea about the features of dogs and cat so we can’t categorize it in dogs and cats. •But it can categorize them according to their similarities, patterns, and can easily categorize the picture into two parts. First part may contain all pics having dogs in it and second part may contain all pics having cats in it. Here you didn’t learn anything before, means no training data or examples. • It allows the model to work on its own to discover patterns and information that was previously undetected. It mainly deals with unlabelled data. • For instance, suppose it is given an image having both dogs and cats which have not seen ever.
- 30. Identification/ Prediction 30 Group of Cat-Dog Group of Cat Group of Dog
- 31. Comparison Chart Parameters Supervised machine learning Unsupervised machine learning Input Data Algorithms are trained using labeled data. Algorithms are used against data which is not labelled Computational Complexity Simpler method computationally complex Accuracy Highly accurate Less accurate 31
- 32. learning algorithms Supervised: Adaline, Madaline Perceptron Back Propagation multilayer perceptrons Radial Basis Function Networks Unsupervised Competitive Learning Kohenen self organizing map Learning vector quantization Hebbian learning
- 33. Deep Learning 33 Deep Learning refers to when many layers are used to build deep networks. State-of-the-art models use hundreds of layers. Deep layers tend to decrease in width. Successive layers transform inputs with two effects: • Compression: each layer is asked to summarize the input in a way that best serves the task. • Extraction: the model succeeds when each layer extracts task-relevant information.
- 34. Smaller to Deeper Network We know it is good to learn a small model. From this fully connected model, do we really need all the edges? Can some of these be shared?
- 35. Deep Learning Model 35
- 36. Pooling Process in ANN Fully Connected Feedforward network Dog or cat Convolution Max Pooling Convolution Max Pooling Flattened repeat many times
- 37. Deep Learning Application 37
- 38. Application Science and medicine: modeling, prediction, diagnosis, pattern recognition Manufacturing: process modeling and analysis Marketing and Sales: analysis, classification, customer targeting Finance: portfolio trading, investment support Banking & Insurance: credit and policy approval Security: bomb, iceberg, fraud detection Engineering: dynamic load schedding, pattern recognition 38