![_big data
Challenges - Respecting all geoscience data
Heterogeneities in Space and Time (Spatio-Temporal) [1]
Rare events
Changing resolution with depth
Amorphous boundaries
Lack of gold-standard ground truth
[1] Karpatne, A., Ebert-Uphoff, I., Ravela, S., Babaie, H. A., & Kumar, V. (2018). Machine learning for the geosciences: Challenges and
opportunities. IEEE Transactions on Knowledge and Data Engineering.](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-10-320.jpg)
![_big data
Opportunites - Particularly towards ML applications in
Exploration Geophysics
Heterogeneities in Space and Time [1]
Developed forward models
[1] Karpatne, A., Ebert-Uphoff, I., Ravela, S., Babaie, H. A., & Kumar, V. (2018). Machine learning for the geosciences: Challenges and
opportunities. IEEE Transactions on Knowledge and Data Engineering.](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-11-320.jpg)
![_neural networks
Artificial Neural Network
“Cells that fire together wire together”
Weight (W) Bias (b)
MCP Neuron [2]
Activation Function (a)
Parameters {W, b}
[2] McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas
immanent in nervous activity. The bulletin of mathematical biophysics,
5(4), 115-133.
data](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-26-320.jpg)
![_neural networks
Artificial Neural Network
“Cells that fire together wire together”
Weight (W) Bias (b)
MCP Neuron [2]
Activation Function (a)
Parameters {W, b}
Loss (Error)
Back Propogation (Gradient Descent) [3]
[2] McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas
immanent in nervous activity. The bulletin of mathematical biophysics,
5(4), 115-133.
[3] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning
representations by back-propagating errors. Nature, 323(6088), 533.
data
error](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-27-320.jpg)
![[3] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning
representations by back-propagating errors. Nature, 323(6088), 533.](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-28-320.jpg)
![_deep neural networks
[4] Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A. R., Jaitly, N., ... & Kingsbury, B. (2012). Deep neural networks for acoustic
modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6), 82-97.
[5] Krizhevsky, A., Sutskever, I. & Hinton, G. (2012) ImageNet classification with deep
convolutional neural networks. In Proc. Advances in Neural Information Processing Systems 25 1090–1098
Speech [4]
Image [5]
Source: https://www.mql5.com/en/articles/1103](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-29-320.jpg)
![_convolutional neural networks
[6] LeCun, Y., Boser, B. E., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W. E., & Jackel, L. D. (1990).
Handwritten digit recognition with a back-propagation network. In Advances in neural information processing
systems (pp. 396-404).
Source: https://cdn-images-1.medium.com/max/1600/0*P9KRZdTs6qwZPkss.png
convolution > compression > dimesnionality reduction > simple aspects > high level features
pixels](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-30-320.jpg)
![[7] Zhu, W., & Beroza, G. C. (2018). PhaseNet: A Deep-
Neural-Network-Based Seismic Arrival Time Picking
Method. arXiv preprint arXiv:1803.03211.
_deep seismic](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-31-320.jpg)
![_deep seismic
Source: https://towardsdatascience.com/generative-adversarial-networks-explained-34472718707a
[8] Mosser, L., Kimman, W., Dramsch, J., Purves, S., De la Fuente Briceño, A., & Ganssle, G. (2018, June). Rapid seismic domain
transfer: Seismic velocity inversion and modeling using deep generative neural networks. In 80th EAGE Conference and Exhibition 2018.](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-32-320.jpg)
![[8] Mosser, L., Kimman, W., Dramsch, J., Purves, S., De la Fuente Briceño, A., & Ganssle, G. (2018, June). Rapid seismic domain transfer: Seismic
velocity inversion and modeling using deep generative neural networks. In 80th EAGE Conference and Exhibition 2018.
model2seis](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-33-320.jpg)
![_references
[1] Karpatne, A., Ebert-Uphoff, I., Ravela, S., Babaie, H. A., & Kumar, V. (2018). Machine learning for the geosciences: Challenges and
opportunities. IEEE Transactions on Knowledge and Data Engineering.
[2] McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical
biophysics, 5(4), 115-133.
[3] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088),
533.
[4] Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A. R., Jaitly, N., ... & Kingsbury, B. (2012). Deep neural networks for acoustic
modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6), 82-97.
[5] Krizhevsky, A., Sutskever, I. & Hinton, G. ImageNet classification with deep
convolutional neural networks. In Proc. Advances in Neural Information Processing Systems 25 1090–1098 (2012)
[6] LeCun, Y., Boser, B. E., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W. E., & Jackel, L. D. (1990). Handwritten digit
recognition with a back-propagation network. In Advances in neural information processing systems (pp. 396-404).
[7] Zhu, W., & Beroza, G. C. (2018). PhaseNet: A Deep-Neural-Network-Based Seismic Arrival Time Picking Method. arXiv preprint
arXiv:1803.03211.
[8] Mosser, L., Kimman, W., Dramsch, J., Purves, S., De la Fuente Briceño, A., & Ganssle, G. (2018, June). Rapid seismic domain
transfer: Seismic velocity inversion and modeling using deep generative neural networks. In 80th EAGE Conference and Exhibition 2018.](https://image.slidesharecdn.com/seminarpresentationv3-190728194520/85/A-scoping-review-of-Machine-Learning-in-Seismic-Geophysics-34-320.jpg)
A scoping review of Machine Learning in Seismic Geophysics
- 1. A scoping review of Machine Learning in Seismic Geophysics Chris Mancuso Graduate Seminar Series
- 2. Machine Learning in Scientific Literature (1990- ) Collected from Web of Science (WOS)
- 3. _terminology Artificial Intelligence (AI) – A computer program that does smart work Machine Learning (ML) – A computer program that can be trained to make predictions Realizes AI Deep Learning – One Machine Learning approach Modeled on human brain (with Artificial Neural Networks) Can learn without training (unsupervised)
- 5. ARTIFICIAL INTELLIGENCE MACHINE LEARNING BRAIN INSPIRED _topology Principal component analysis Random Forest Spiking NN Random Forest Fuzzy Logic Automation Deep Learning Natural Language Processing Symbolic Logic Kernel Methods
- 6. ML by Research Area (WOS Data)
- 7. _big data Large datasets 5 V’s: volume, velocity, variety, veracity, value 2 sources of data in geophysics: Measured from instruments Modeled from physics
- 8. Source: http://ds.iris.edu/ds/nodes/dmc/earthscope/usarray/ USArray - Seismometers
- 9. Liu, E. D., & Prescop, T. (2011, April). Optimization of e-beam landing energy for EBDW. In Alternative Lithographic Technologies III (Vol. 7970, p. 79701S). International Society for Optics and Photonics. Monte Carlo Simulation of electon beam in Silicon target
- 10. _big data Challenges - Respecting all geoscience data Heterogeneities in Space and Time (Spatio-Temporal) [1] Rare events Changing resolution with depth Amorphous boundaries Lack of gold-standard ground truth [1] Karpatne, A., Ebert-Uphoff, I., Ravela, S., Babaie, H. A., & Kumar, V. (2018). Machine learning for the geosciences: Challenges and opportunities. IEEE Transactions on Knowledge and Data Engineering.
- 11. _big data Opportunites - Particularly towards ML applications in Exploration Geophysics Heterogeneities in Space and Time [1] Developed forward models [1] Karpatne, A., Ebert-Uphoff, I., Ravela, S., Babaie, H. A., & Kumar, V. (2018). Machine learning for the geosciences: Challenges and opportunities. IEEE Transactions on Knowledge and Data Engineering.
- 12. _bibliometric methods Goal: Intersection of ML and Geo Deep learning in seismic geophysics Accessed records from WOS (Clarivate Analytics) Limited to journal articles in english published on or after 1980 Keyword Mining (top 250 in ML) Document Types Records % of 303845 ARTICLE 303845 100 PROCEEDINGS PAPER 21267 6.999 BOOK CHAPTER 89 0.029 RETRACTED PUBLICATION 40 0.013 DATA PAPER 25 0.008 EARLY ACCESS 5 0.002
- 13. 2018 1980 Human Genome Project ML by Discipline Area over time (WOS Data)
- 14. _methods_cont Accessed records from WOS (Clarivate Analytics) Limited to journal articles in english published on or after 2000 Refined search incrementally Pre-processed Data before analysis (R)
- 15. N = 3065 Articles
- 16. Research Areas
- 18. Research Methods
- 19. Research Methods
- 20. N = 557 Articles
- 21. Research Areas
- 22. Research Areas
- 23. Internal search engine system Returned 1197 patent applications and grants filed after year 2000 _USPTO search 0 100 200 300 400 500 600 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 No.Patents Year USPTO Patent Applications and Grant
- 24. _literature review Concluded from meta analysis: Growth in application of ML in seismic data interpretation and processing Rock properties (porosity, density), event picking, noise attenuation, velocity analysis Applied in both earthquake seismology and seismic imaging (tomography, inversion) Represent a growth in observation data Recent acute movement to artificial neural network (ANN) solutions Most recent inclusion of Deep Neural Networks (Deep Learning) Convolutional Neural Networks (CNN) adopted in 2018
- 25. Moving from broader scope lit review to focused discussion advancement of the methodology used in artificial neural networks (ANN) their application to seismic data processing _neural networks Source: https://en.wikipedia.org/wiki/Artificial_neural_network, https://medium.com/ml- algorithms/neural-networks-for-decision-boundary-in-python-b243440fb7d1,
- 26. _neural networks Artificial Neural Network “Cells that fire together wire together” Weight (W) Bias (b) MCP Neuron [2] Activation Function (a) Parameters {W, b} [2] McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4), 115-133. data
- 27. _neural networks Artificial Neural Network “Cells that fire together wire together” Weight (W) Bias (b) MCP Neuron [2] Activation Function (a) Parameters {W, b} Loss (Error) Back Propogation (Gradient Descent) [3] [2] McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4), 115-133. [3] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533. data error
- 28. [3] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533.
- 29. _deep neural networks [4] Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A. R., Jaitly, N., ... & Kingsbury, B. (2012). Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6), 82-97. [5] Krizhevsky, A., Sutskever, I. & Hinton, G. (2012) ImageNet classification with deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems 25 1090–1098 Speech [4] Image [5] Source: https://www.mql5.com/en/articles/1103
- 30. _convolutional neural networks [6] LeCun, Y., Boser, B. E., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W. E., & Jackel, L. D. (1990). Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems (pp. 396-404). Source: https://cdn-images-1.medium.com/max/1600/0*P9KRZdTs6qwZPkss.png convolution > compression > dimesnionality reduction > simple aspects > high level features pixels
- 31. [7] Zhu, W., & Beroza, G. C. (2018). PhaseNet: A Deep- Neural-Network-Based Seismic Arrival Time Picking Method. arXiv preprint arXiv:1803.03211. _deep seismic
- 32. _deep seismic Source: https://towardsdatascience.com/generative-adversarial-networks-explained-34472718707a [8] Mosser, L., Kimman, W., Dramsch, J., Purves, S., De la Fuente Briceño, A., & Ganssle, G. (2018, June). Rapid seismic domain transfer: Seismic velocity inversion and modeling using deep generative neural networks. In 80th EAGE Conference and Exhibition 2018.
- 33. [8] Mosser, L., Kimman, W., Dramsch, J., Purves, S., De la Fuente Briceño, A., & Ganssle, G. (2018, June). Rapid seismic domain transfer: Seismic velocity inversion and modeling using deep generative neural networks. In 80th EAGE Conference and Exhibition 2018. model2seis
- 34. _references [1] Karpatne, A., Ebert-Uphoff, I., Ravela, S., Babaie, H. A., & Kumar, V. (2018). Machine learning for the geosciences: Challenges and opportunities. IEEE Transactions on Knowledge and Data Engineering. [2] McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4), 115-133. [3] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533. [4] Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A. R., Jaitly, N., ... & Kingsbury, B. (2012). Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6), 82-97. [5] Krizhevsky, A., Sutskever, I. & Hinton, G. ImageNet classification with deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems 25 1090–1098 (2012) [6] LeCun, Y., Boser, B. E., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W. E., & Jackel, L. D. (1990). Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems (pp. 396-404). [7] Zhu, W., & Beroza, G. C. (2018). PhaseNet: A Deep-Neural-Network-Based Seismic Arrival Time Picking Method. arXiv preprint arXiv:1803.03211. [8] Mosser, L., Kimman, W., Dramsch, J., Purves, S., De la Fuente Briceño, A., & Ganssle, G. (2018, June). Rapid seismic domain transfer: Seismic velocity inversion and modeling using deep generative neural networks. In 80th EAGE Conference and Exhibition 2018.
- 35. Questions?
Editor's Notes
- #3: Kick things off with a chart All ML articles from WOS since 1990 in all scietific disciplines Shows exponential increase in the last 6 years Relevant tool in many fields and would like to discern its relevance in seismic GP Especially because i am looking into methods-based research involving this
- #4: A few quick terms to get everyone on the same page for this presentation Robotic arms, search engines, russian chat bots etc
- #5: Another way to think of those terms is like a nested russian doll
- #6: Many methods fall within each layer The one i will be eventually focusing on is deep learning
- #7: This tree plot represents that same graph from earlier split into research areas (top 25) Ive removed computer science (the obvious one) as I am demonstrantrating the multi disciplinary approach to a wide variety of fields Geology Remote sensing
- #8: A point I want to bring across is that data drives innovation and adoption of methods in this field Can not talk about ML or AI without Big Data Because these dataset are too large and complex for a person to reasonably interpret them
- #9: Example of instrument acquired data USGS earthquake sensors across USA
- #10: Example of model driven data Electron beam impacting a material Monte carlo simulation
- #26: Decision boundaries
- #27: Error function such as Least Squares PDE shown to be efficiently solved using chain rule
- #28: Error function such as Least Squares PDE shown to be efficiently solved using chain rule
- #30: This joint paper from the major speech recognition laboratories, summarizing the breakthrough achieved with deep learning on the task of phonetic classification for automatic speech recognition, was the first major industrial application of deep learning.
- #31: This is the first paper on convolutional networks trained by backpropagation for the task of classifying low-resolution images of handwritten digits.