Inversion_Parmetrization_under_det_problem.pdf
- 1. Slides from Ved Lekic INVERSE THEORY 1
- 2. 2
- 3. Parameterization u What quantities (parameters) should we consider? u Consider pairs of quantities, such as measurements (y) made at conditions (x) Measurements (data) Conditions (independent variable)
- 4. Parameterization u Physical model may provide insight into how to parameterize a problem u Sometimes we seek “simplest” model (fewest parameters) that can fit data Measurements (data) Conditions (independent variable)
- 5. Law of parsimony – a useful heuristic u Preference for “simpler” models is often justified by invoking “Occam’s Razor”. u e.g. “It is vain to do with more what you can do with less.” – some 13th century monk u Karl Popper argues that simplest models are preferable because they can be more easily falsified u "I remember my friend Johnny von Neumann used to say, with four parameters I can fit an elephant, and with five I can make him wiggle his trunk.“ -- Enrico Fermi, Nature (2004). Measurements (data) Conditions (independent variable)
- 6. Under-determined problem u With 1 data point, we cannot uniquely determine slope and y- intercept u Complete trade-off between m1 and m2! Measurements (data) Conditions (independent variable)
- 7. Even-determined problem u When we have 2 data points, we can uniquely determine m1 and m2 Measurements (data) Conditions (independent variable)
- 8. Over-determined problem u With >2 points, the line is not guaranteed to pass through all points. u What do we choose? u What’s the BEST line … objectively speaking? Measurements (data) Conditions (independent variable)
- 9. The “objective” function u We introduce an objective (a.k.a. cost, misfit) function that is minimum when our line provides the “best” fit u i.e. we minimize εj’s Measurements (data) Conditions (independent variable) ε4 ε3 ε2 ε1
- 10. Choices of Norm for objective functions u L-1 u Sum of absolute values of residuals u Tends to ignore outliers u L-2 u Sum of squared residuals u Tries to fit outliers u “Least-squares” u L-∞ u Maximum residual u Fit is “never terrible” but usually not good, either u “Minimax” φ 𝑚 = $ !"# $! 𝑑! − 𝑔!(𝑚) φ 𝑚 = 𝑚𝑎𝑥 𝑑! − 𝑔!(𝑚) φ 𝑚 = $ !"# $! 𝑑! − 𝑔!(𝑚) %
- 11. What about data uncertainty? u If error of one measurement is expected to be larger than that of another, then the line need not pass close to it, nor should the objective function count it as much as more accurate measurements Measurements (data) Conditions (independent variable)
- 12. What about data uncertainty? u If data error is normally distributed with variance σj 2 then the correct objective function is L-2 u Each squared residual gets weighted by variance: Measurements (data) Conditions (independent variable) φ 𝑚 = $ !"# $! 𝑑! − 𝑔!(𝑚) % 𝜎! %
- 13. u Matrix notation: Fitting a line to some points u Linear problem: u 𝑦! = 𝑚"𝑥! + 𝑚# Y - Measurements (data) X - Conditions (independent variable) φ 𝑚 = $ !"# $! 𝑑! − 𝑔!(𝑚) % 𝜎! % 𝑥# 1 𝑥% 1 𝑥& 1 𝑥' 1 𝑚! 𝑚" 𝑦! 𝑦" 𝑦# 𝑦$ = 𝐶! "#/% 𝑑 = 𝐶! "#/% 𝐺𝑚 ⁄ # (" 0 0 ⁄ # (# 0 0 0 0 0 0 0 0 ⁄ # ($ 0 0 ⁄ # (% 𝜑 = (𝑑 − 𝐺𝑚)& 𝐶! "# (𝑑 − 𝐺𝑚) ⁄ # (" 0 0 ⁄ # (# 0 0 0 0 0 0 0 0 ⁄ # ($ 0 0 ⁄ # (%
- 14. u Matrix notation: u Minimize φ with respect to m u Condition is satisfied when: u “Least squares” estimate of model parameters: mest Fitting a line to some points 𝐶! "#/% 𝑑 = 𝐶! "#/% 𝐺𝑚 𝜑 = (𝑑 − 𝐺𝑚)& 𝐶! "# (𝑑 − 𝐺𝑚) '( ') = 0 𝑚*+, = (𝐺& 𝐶! "# 𝐺)"# 𝐺& 𝐶! "# 𝑑 u Linear problem: u 𝑦! = 𝑚"𝑥! + 𝑚# Y - Measurements (data) X - Conditions (independent variable) φ 𝑚 = $ !"# $! 𝑑! − 𝑔!(𝑚) % 𝜎! %
- 15. How confident are we in mest? u We can evaluate the uncertainty of model parameters using the posterior model covariance matrix: " 𝐶0 u But first, let’s build some intuition… Conditions Measurements Which scenario – left or right – will yield a less uncertain estimate of slope? Measurements Conditions
- 16. How confident are we in mest? u We can evaluate the uncertainty of model parameters using the posterior model covariance matrix: . 𝐶) u But first, let’s build some intuition… Conditions Measurements Which scenario – left or right – will yield a less uncertain estimate of slope? Measurements Conditions BETTER! à Objective function is more affected by slope variations when data uncertainty is small à more accurate data tends to yield more reliable estimates of parameters
- 17. How confident are we in mest? u We can evaluate the uncertainty of model parameters using the posterior model covariance matrix: . 𝐶) u But first, let’s build some intuition… Conditions Measurements Which scenario – left or right – will yield a less uncertain estimate of slope? Measurements Conditions BETTER! à Predictions of widely spaced (in x) points are more sensitive to slope variations à estimates of parameters to which you are more sensitive tend to be better!
- 18. How confident are we in mest? u We can evaluate the uncertainty of model parameters using the posterior model covariance matrix: . 𝐶) 𝑚*+, = (𝐺& 𝐶! "# 𝐺)"# 𝐺& 𝐶! "# 𝑑 " 𝐶0= (𝐺& 𝐶! "# 𝐺)"# " 𝐶0= 𝜎+-./* % 𝜎+-./*𝜎01,𝜌 𝜎+-./*𝜎01,𝜌 𝜎01, % Variance of slope estimate Variance of y- intercept estimate Correlation between estimates of slope and y-intercept à how much parameters trade-off!
- 19. Limitation of ! 𝐶! u Posterior model covariance matrix only describes the generalized Gaussian distribution around the optimal model 𝑚*+, = (𝐺& 𝐶! "# 𝐺)"# 𝐺& 𝐶! "# 𝑑 " 𝐶0= (𝐺& 𝐶! "# 𝐺)"# " 𝐶0= 𝜎+-./* % 𝜎+-./*𝜎01,𝜌 𝜎+-./*𝜎01,𝜌 𝜎01, % Variance of slope estimate Variance of y- intercept estimate Correlation between estimates of slope and y-intercept à how much parameters trade-off!
- 20. Schematic representation data model LINEAR PROBLEM d = Gm m est after Tarantola, 2005 Low High Low Likelihood
- 21. Types of inverse problems Linear – least squares works and ! 𝐶) is accurate Linearizable around starting model ! 𝐶) is probably OK Tarantola, 2005
- 22. Non-linear à multi-modal posterior on m, ! 𝐶) is woefully invalid! • Don’t use least-squares! SAMPLING!J Linearizable around most likely model • Must iterate! • ! 𝐶) might be OK Tarantola, 2005
- 23. nothing will work! 𝐺𝑜𝑜𝑑 𝑙𝑢𝑐𝑘! Tarantola, 2005
- 24. data model d = Gm Perfect Inverse Problem m est u Unrealistic illustration: Perfect data, perfectly known G, and all model parameters reconstructed perfectly.
- 25. Uncertainty about relationship between data and model data model d = Gm Modeling Uncertainty Model uncertainty!
- 27. Effect of forward modeling error u Consequences of incorrectly computing g(m) can be significant: u Equivalent to errors to data thereby increasing model estimate uncertainty Lekic and Romanowicz, 2011 u Can systematically bias the model estimate " 𝐶0= (𝐺& 𝐶! + 𝐶& "# 𝐺)"#
- 28. Common complication… u Often, real-world problems are mixed-determined u Some parameters are well constrained u Other parameters are poorly or completely un-constrained u Sometimes it happens that det(𝐺1 𝐶2 34 𝐺) = 0 and therefore (𝐺1𝐶2 34𝐺))# – a.k.a. G-g – does NOT exist (i.e. there are an insufficient number of linearly independent constraints to determine the unknowns) u What do we do in that case?