AI in Physics - University of Washington, Jan 2024
Download as PPTX, PDF0 likes184 views
The document outlines Peter Morgan's background in physics and AI, emphasizing his motivation to integrate AI into scientific research to accelerate discoveries. It discusses the exponential growth of data and AI capabilities and showcases various applications of AI in different fields of science, particularly in nuclear physics. The document also highlights ongoing research collaborations and available resources for further exploration of AI in scientific contexts.
AI in Physics - University of Washington, Jan 2024
- 1. AI in Physics University of Washington Incubator for Quantum Simulation – IQuS Peter Morgan www.deeplp.com UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 2. About Me • MSc (physics), University of Auckland • PhD (physics), UMass Amherst with Barry Holstein (ABD) • Founded company • Then IT Consultant (10 years) • Academia again as Research Associate – 3 years studying neutrino double beta decay (mass) • AI Consultant (last 10 years) • Founded Deep Learning Partnership • AI consulting mostly for businesses (also govt, education) • Some quantum computing consulting (bit early for commercial application) UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 3. Motivation • Contribute to accelerating science • Applying AI to science – very impactful • Due to my physics background, I am motivated to use AI to solve science, as well as business, problems • Naturally inclined to combine AI and science • Hopefully provide UW Physics with motivation and research starting points UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 4. AI Hierarchy & Terminology UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024 • We will use these three terms somewhat interchangeably (even though we shouldn’t) • Deep learning = ANNs (neural networks) • AI – three categories 1. Narrow – ANI* 2. General – Human level, AGI** 3. Super – ASI • Beyond human • Narrow ASI* • General ASI** * Solved **Not solved
- 5. So why AI now? • Data, hardware and AI models have all been increasing exponentially over the past 30 years • We are now at trillions of tokens (words), trillions of model parameters, and Exaflops of compute (Exa = 10^18) • In 2012 we were at millions of tokens, millions of parameters and Teraflops (Tera = 10^12) • So a factor of ~million in all three dimensions ! • Recall human brain has ~100 billion neurons & ~1000 trillion synapses • Uses ~ 1million times less energy but contains a lot less information (memory) • LLMs contain & have processed virtually the whole of the Internet • So we are in interesting times – let’s make the most of AI! UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 6. ANN Model Growth* * Data sets and hardware are on similar exponentials (but also cost) UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 7. Intelligent Capabilities Emerge with Scale https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 8. AI Models • Proprietary • GPT-4, Copilot, Gemini, Claude, Cohere, … • OpenAI, Microsoft, Google, Anthropic, Cohere, … • Access via API calls for a cost • Best performance so far • Open source • Mistral, Falcon, Llama-2, Bloom, … • Companies & university research groups • Download model weights for free • Lower performance than proprietary but catching up • See leaderboards • e.g., https://huggingface.co/spaces/lmsys/chatbot-arena- leaderboard UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 9. How do they work? (we don’t completely know) UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 10. ANNs – High Level • General Purpose Technology • Discriminative vs Generative • Many types of architectures and approaches • Use back propagation • Data can be multimodal • Some mysteries remain, e.g., emergence • Lacking a complete mathematical theory • Complex statistical systems • See references at end UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 11. Landscape of Generative AI GANs Variational Autoencoders Energy-based Models Normalizing Flows Transformers Diffusion Models UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 12. Active Areas of AI Research • Open vs closed source models • Large vs small (trillions vs billions of parameters) • Architectures • Encoder only (autoencoder) • Decoder only (autoregressive) • Encoder-Decoder (seq2seq) • Compression – quantization, distillation, etc. • Fine-tuning custom models • Multimodality • Retrieval (RAG) • Reasoning • AI Agents • Scaling Laws • Emergence – scale is all you need • AGI (human-like intelligence) UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 13. AI can be used in all of science • Cosmology • Quantum gravity • Particle physics • Nuclear physics • Materials science • Chemistry • Molecular dynamics • Biology UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 14. Nuclear Physics Example – NNPDF Collaboration • Neural Networks for Parton Distribution Function • “The NNPDF collaboration determines the structure of the proton using contemporary methods of artificial intelligence” • Use data from many particle experiments, including RHIC, Tevatron, LHC • Global collaboration going back to 2004 • Homepage: https://nnpdf.mi.infn.it • Overview paper: Ball, R. et al, The Path to Proton Structure at One-Percent Accuracy, 31 May 2022, https://arxiv.org/abs/2109.02653 • Opensource code available: https://github.com/NNPDF/nnpdf • NNPDF collaboration papers (23 so far): https://arxiv.org/search/hep- ph?searchtype=author&query=NNPDF+Collaboration UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 15. UW IQuS – AI in Physics UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 16. UW IQuS – AI in Physics UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024 Can expect lots more data, more capable AI frameworks, plus more powerful hardware
- 17. References © Peter Morgan 17 Jan 2024 UW IQuS – AI in Physics References
- 18. AI Reasoning Capabilities • Trinh, T. et al, Solving Olympiad geometry without human demonstrations, 17 Jan2024, https://www.nature.com/articles/s41586-023-06747-5 Google Deepmind • Sun, J. et al, A Survey of Reasoning with Foundation Models, 26 Dec 2023, https://arxiv.org/abs/2312.11562v4 • Gurnee, W. & M. Tegmark, Language Models Represent Space and Time, 14 Dec 2023, https://arxiv.org/abs/2310.02207, MIT • Yao, S. et al, Tree of Thoughts: Deliberate Problem Solving with Large Language Models, 3 Dec 2023, https://arxiv.org/abs/2305.10601, Google Deepmind • Shi, W. et al, In-Context Pretraining: Language Modeling Beyond Document Boundaries, 30 Nov 2023, https://arxiv.org/abs/2310.10638, UW CS/ML & Allen Institute • Sumers, T. et al, Cognitive Architectures for Language Agents, 27 Sept 2023, https://arxiv.org/abs/2309.02427, Princeton • Greyling, C. LangChain, LangSmith & LLM Guided Tree-of-Thought, 13 Sept 2023, https://cobusgreyling.medium.com/langchain-langsmith-llm-guided-tree-of-thought-47a2cd5bcfca • Bubeck, S. et al, Sparks of Artificial General Intelligence: Early experiments with GPT-4, April 2023, https://arxiv.org/abs/2303.12712, Microsoft Research • Paranjape, B. et al, ART: Automatic multi-step reasoning and tool-use for large language models, 16 Mar 2023, https://arxiv.org/abs/2303.09014, UW, MSR & Allen Institute UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 19. AI4Science Research Groups All the major cloud providers are investing heavily in AI for science: Google Deepmind • Materials science: Merchant, A. et al, Scaling deep learning for materials discovery, 29 Nov 2023, https://www.nature.com/articles/s41586-023-06735-9 • Fusion: Degrave, J. et al, Magnetic control of tokamak plasmas through deep reinforcement learning, 16 Feb 2022, https://www.nature.com/articles/s41586-021-04301-9 • Mathematics: Romera-Paredes, B. et al, Mathematical discoveries from program search with large language models, 14 Dec 2023, https://www.nature.com/articles/s41586-023-06924-6 • Computer science: Mankowitz, D. et al, Faster sorting algorithms discovered using deep reinforcement learning, 7 June 2023, https://www.nature.com/articles/s41586-023-06004-9 • Biology: Jumper, J. et al, Highly accurate protein structure prediction with AlphaFold, 15 July 2021, https://www.nature.com/articles/s41586-021-03819-2 MSR • AI4Science: https://www.microsoft.com/en-us/research/lab/microsoft-research-ai4science/ • PNNL Collaboration: https://www.pnnl.gov/pnnl-microsoft-collaboration AWS, https://www.amazon.science/about ByteDance, https://www.forbes.com/sites/alexandralevine/2024/01/02/tiktok-bytedance-pharmaceuticals- drug-discovery-science-biology-chemistry-ai-china/?sh=5d5072615087 UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 20. How is AI being used in nuclear physics? • Zhang, X. et al, Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems, 15 Nov 2023, https://arxiv.org/abs/2307.08423 • Matchev, K.T., et al, Seeking Truth and Beauty in Flavor Physics with Machine Learning, 31 Oct 2023, https://arxiv.org/abs/2311.00087 • Cranmer, K. et al, Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics, 3 Sept 2023, https://arxiv.org/abs/2309.01156 • Abbott, R. et al, Normalizing flows for lattice gauge theory in arbitrary space-time dimension, 3 May 2023, https://arxiv.org/abs/2305.02402 • Liu, Z. et al, GenPhys: From Physical Processes to Generative Models, 5 April 2023, https://arxiv.org/abs/2304.02637 • He, Y-H. et al, Machine Learning in Physics and Geometry, 30 Mar, 2023, https://arxiv.org/abs/2303.12626 • Butter, A. et al, Machine Learning and LHC Event Generation, 28 Dec 2022, https://arxiv.org/abs/2203.07460 • Liu, Z. et al, AI Poincaré 2.0: Machine Learning Conservation Laws from Differential Equations, 30 Oct 2022, https://arxiv.org/abs/2203.12610 • Shanahan, P. et al, Snowmass 2021 CompF03 Topical Group Report: Machine Learning, 15 Sept 2022, https://arxiv.org/abs/2209.07559 • Boehnlein, A. et al, Machine Learning in Nuclear Physics, 2 May 2022, https://arxiv.org/abs/2112.02309 <-- review paper UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 21. Historic Papers of NNs in Particle Physics • Denby, B., Neural networks and cellular automata in experimental high energy physics, 3(49), June 1988! https://www.sciencedirect.com/science/article/abs/pii/0010465588900045 • Cutts, D. et al, Applications of neural networks in high energy physics, Aug 1990, https://www.osti.gov/biblio/5954034 • Lonnblad, L. et al, Using neural networks to identify jets, Nucl. Phys. B349 (1991) 675–702 • Kanev, Y.A., Application of neural networks and genetic algorithms in high- energy physics, UMI-99-05968 • Forte, S. et al, Neural network parametrization of deep-inelastic structure functions, JHEP 05 (2002) 062, https://arxiv.org/abs/hep-ph/0204232 UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 22. ML and Quantum Computing • Castelvecchi, D., The AI–quantum computing mash-up: will it revolutionize science?, 2 Jan 2024, https://www.nature.com/articles/d41586-023-04007-0 • Bausch, J. et al, Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network, 9 Oct 2023, https://arxiv.org/abs/2310.05900 • Machine Learning Aids Classical Modeling of Quantum Systems, Quanta Magazine, 14 Sept 2023, https://www.quantamagazine.org/machine- learning-aids-classical-modeling-of-quantum-systems-20230914 • Huang, H-Y. et al, Learning to predict arbitrary quantum processes, 15 April 2023, https://arxiv.org/abs/2210.14894 • Lewis, L. et al, Improved machine learning algorithm for predicting ground state properties, 30 Jan 2023, https://arxiv.org/abs/2301.13169 • Moon, H. et al, Machine learning enables completely automatic tuning of a quantum device faster than human experts, 8 Jan 2020, https://arxiv.org/abs/2001.02589 UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 23. Books & Online Resources • Erdmann, M. et al, Deep Learning for Physics Research, World Scientific, 2021 • Tanaka, A. et al, Deep Learning and Physics, Springer Link, 2021 • Roberts, D. et al, Principles of Deep Learning Theory, CUP, 2021, or arXiv, https://arxiv.org/abs/2106.10165 • Simon Prince, Understanding Deep Learning, MIT Press, Dec 2023 (also free online) • A Living Review of Machine Learning for Particle Physics, https://iml- wg.github.io/HEPML-LivingReview/ • AI for Science, https://ai4sciencecommunity.github.io • Undermind, an AI enabled research paper search engine, undermind.ai UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 24. Some Physics ML Research Groups • MIT IAIFI, https://iaifi.org • CERN Openlab, https://openlab.cern • LANL, https://discover.lanl.gov/news/1017-ai-machine-learning/ • FNAL, https://computing.fnal.gov/artificial-intelligence/ • PNNL, https://www.pnnl.gov/artificial-intelligence • Argonne, https://www.alcf.anl.gov/alcf-ai-testbed • Flatiron, https://www.simonsfoundation.org/machine-learning-at-the- flatiron-institute/ • NNPDF, https://nnpdf.mi.infn.it • MSU FRIB, https://frib.msu.edu UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 25. AI in Physics Workshops • Aspen Winter Conference Jan 2024, Fields, Strings & Deep learning, https://indico.cern.ch/event/1299185/ • MIT IAIFI Summer Workshops (every year), https://iaifi.org/summer-workshop • CERN 2023, AI4Science Workshop, https://indico.cern.ch/event/1326114/ • NeurIPS 2023, AI for Scientific Discovery: From Theory to Practice, https://ai4sciencecommunity.github.io/neurips23.html • ICML 2022, AI for Science: Theories and Foundations, https://ai4sciencecommunity.github.io/neurips23.html • UCLA IPAM 2019, Machine Learning for Physics and the Physics of Learning, http://www.ipam.ucla.edu/programs/workshops/machine-learning-for- physics-and-the-physics-of-learning-tutorials/ UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 26. Further Resources Physicists (check out their papers) • Max Tegmark – MIT • Kyle Cranmer – University of Wisconsin-Madison • Danilo Rezende – Google Deepmind AI Research in Seattle* • UW Computer Science, e.g., Luke Zettlemoyer • Allen Institute for AI, e.g., Noah Smith • Microsoft Research, Redmond *Collaboration is fruitful for both physicists and AI researchers UW IQuS – AI in Physics © Peter Morgan 17 Jan 2024
- 27. Questions & Discussion © Peter Morgan 17 Jan 2024 UW IQuS – AI in Physics