Mathematics for machine learning calculus formulasheet
2 likes2,180 views
This document provides formulas and definitions for concepts in multivariate calculus and machine learning. It defines the derivative and lists time-saving rules like the sum, product, and chain rules. It also defines the Jacobian and Hessian matrices, Taylor series, optimization techniques like Newton-Raphson and gradient descent, and concepts like Lagrange multipliers and least squares minimization. The document is a formula sheet for a course on mathematics for machine learning.
Mathematics for machine learning calculus formulasheet
- 1. Mathematics for Machine Learning Multivariate Calculus Formula sheet Dr Samuel J. Cooper Prof. David Dye Dr A. Freddie Page Definition of a derivative f (x) = df(x) dx = lim ∆x→0 f(x + ∆x) − f(x) ∆x Time saving rules - Sum Rule: d dx (f(x) + g(x)) = d dx (f(x)) + d dx (g(x)) - Power Rule: f(x) = axb f (x) = abx(b−1) - Product Rule: A(x) = f(x)g(x) A (x) = f (x)g(x) + f(x)g (x) - Chain Rule: If h = h(p) and p = p(m) then dh dm = dh dp × dp dm - Total derivative: For the function f(x, y, z, ...), where each variable is a function of parameter t, the total derivative is df dt = ∂f ∂x dx dt + ∂f ∂y dy dt + ∂f ∂z dz dt + . . . Derivatives of named functions ∂ ∂x 1 x = − 1 x2 (1) ∂ ∂x sin x = cos x (2) ∂ ∂x cos x = − sin x (3) ∂ ∂x exp x = exp x (4) Derivative structures f = f(x, y, z) - Jacobian: Jf = ∂f ∂x , ∂f ∂y , ∂f ∂z - Hessian: Hf = ∂2f ∂x2 ∂2f ∂x∂y ∂2f ∂x∂z ∂2f ∂y∂x ∂2f ∂y2 ∂2f ∂y∂z ∂2f ∂z∂x ∂2f ∂z∂y ∂2f ∂z2 1
- 2. Taylor Series - Univariate: f(x) = f(c) + f (c)(x − c) + 1 2 f (c)(x − c)2 + ... = ∞ n=0 f(n) (c) n! (x − c)n - Multivariate: f(x) = f(c) + Jf (c)(x − c) + ... 1 2 (x − c)t Hf (c)(x − c) + ... Optimization and Vector Calculus - Newton-Raphson: xi+1 = xi − f(xi) f (xi) - Grad: f = ∂f ∂x ∂f ∂z ∂f ∂z - Directional Gradient: f.ˆr - Gradient Descent: sn+1 = sn − γ f - Lagrange Multipliers λ: f = λ g - Least Squares - χ2 minimization: χ2 = n i (yi − y(xi; ak))2 σi criterion: χ2 = 0 anext = acur − γ χ2 = acur + γ n i (yi − y(xi; ak)) σi ∂y ∂ak 2