AD3501 - DL Unit-1 PPT.pptx python syllabus
- 3. UNIT I - DEEP NETWORKS BASICS 1.1 Linear Algebra: Scalars, Vectors, Matrices, tensors Linear Algebra for Deep Learning: The Math behind every deep learning program. Deep Learning is a subdomain of machine learning, concerned with the algorithm which imitates the function and structure of the brain called the artificial neural network. Linear algebra is a form of continuous rather than discrete mathematics, many computer scientists have little experience with it. A good understanding of linear algebra is essential for understanding and working with many machine learning algorithms, especially deep learning algorithms. A linear equation is an equation in which the highest power of the variable is always 1. It is also known as a one-degree equation. The standard form of a linear equation in one variable is of the form Ax + B = 0. Here, x is a variable, A is a coefficient and B is constant.
- 4. When confined to smaller levels, everything is math behind deep learning. So it is essential to understand basic linear algebra before getting started with deep learning and programming it. Scalars Scalars are single numbers and are an example of a 0th-order tensor. The notation x states that x is ∈ ℝ a scalar belonging to a set of real-values numbers, ℝ
- 5. Few built-in scalar types are int, float, complex, bytes, Unicode in Python. In In NumPy a python library, there are 24 new fundamental data types to describe different types of scalars. Vectors Vectors are ordered arrays of single numbers and are an example of 1st-order tensor. fragments of objects known as vector spaces. Matrices Matrices are rectangular arrays consisting of numbers and are an example of 2nd-order tensors. If m and n are positive integers, that is m, n then the m×n matrix contains m*n numbers, with m rows and n ∈ ℕ columns. The full m×n matrix can be written as:
- 7. Tensors The more general entity of a tensor encapsulates the scalar, vector and the matrix. It is sometimes necessary — both in the physical sciences and machine learning — to make use of tensors with order that exceeds two. We use Python libraries like tensorflow or PyTorch in order to declare tensors, rather than nesting matrices.
- 8. 1.2 Probability Distributions Probability Distribution is basically the set of all possible outcomes of any random experiment or event. Different Types of Probability Distributions: ● Discrete Probability Distributions for discrete variables ● Cumulative Probability Distribution for continuous variables
- 9. 1.2 Probability Distributions Types of distributions: Common distributions used in deep learning include: ● Normal distribution (bell-shaped curve): For continuous outputs, like predicting house prices. ● Bernoulli distribution (binary outcomes): For classification tasks, like image recognition (cat vs. dog). ● Categorical distribution (multiple categories): When there are more than two classes, like recognizing different types of flowers.
- 10. 1.2 Probability Distributions Binary outcomes formula Categorical Distribution Formula
- 17. Types of Gradient Descent 1. Batch Gradient Descent: Batch gradient descent (BGD) is used to find the error for each point in the training set and update the model after evaluating all training examples. This procedure is known as the training epoch. In simple words, it is a greedy approach where we have to sum over all examples for each update. 2. Stochastic gradient descent Stochastic gradient descent (SGD) is a type of gradient descent that runs one training example per iteration. Or in other words, it processes a training epoch for each example within a dataset and updates each training example's parameters one at a time. 3. MiniBatch Gradient Descent: Mini Batch gradient descent is the combination of both batch gradient descent and stochastic gradient descent. It divides the training datasets into small batch sizes then performs the updates on those batches separately Challenges with the Gradient Descent 1. Local Minima and Saddle Point, 2. Vanishing and Exploding Gradient
- 18. 1.4 Machine Learning Basics Machine learning (ML) is a branch of artificial intelligence (AI) and computer science that focuses on the using data and algorithms to enable AI to imitate the way that humans learn, gradually improving its accuracy.
- 19. 1.4 Machine Learning Basics (Capacity Overfitting and Underfitting) 1.4.1 Overfitting
- 20. 1.4 Machine Learning Basics (Capacity Overfitting and Underfitting) 1.4.1 Overfitting
- 21. 1.4 Machine Learning Basics (Capacity Overfitting and Underfitting) 1.4.2 Underfitting
- 22. 1.4 Machine Learning Basics (Capacity Overfitting and Underfitting High Variance Low Variance, Low Bias High Bias
- 23. 1.4 Machine Learning Basics (Capacity Overfitting and Underfitting)
- 24. 1.7 Bias and Variance 1.7
- 25. 1.7 Bias and Variance
- 26. 1.8 Deep Neural Network Single Perceptron:
- 27. 1.8 Deep Neural Network Multi-Layer Perceptron(MLP):
- 28. 1.8 Deep Neural Network Multi-Layer Perceptron(MLP): Feed Forward Network https://www.youtube.com/watch?v=eOtGPlAS6Yg
- 29. 1.8 Deep Neural Network Multi-Layer Perceptron(MLP): Back Propagation https://www.youtube.com/watch?v=tUoUdOdTkRw