ChatGPT Optional Anlysis with significant
- 1. A Brief Intro to ChatGPT Dr. Josh Locklair
- 2. 2 What is ChatGPT? •Large Language Model (LLM) •Predicts text which should follow from a given prompt using probabilistic methods •Machine learning model trained on massive amounts of text ChatGPT Dr. Josh Locklair
- 3. 3 What is ChatGPT? ChatGPT Dr. Josh Locklair
- 4. 4 GPT-3 •GPT-3 Language Model •499 Billion tokens (words) drawn from the web, books, and Wikipedia •Trained to predict the next word in a sequence ChatGPT Dr. Josh Locklair
- 5. 5 GPT-3 ChatGPT Dr. Josh Locklair Source: https://dugas.ch/artificial_curiosity/GPT_architecture.html
- 6. 6 GPT-3 •Input sequence •Up to 2048 tokens (words encoded as numeric values) •Encoding allows word meanings to be compared (e.g., similar words have similar numeric values) ChatGPT Dr. Josh Locklair Source: https://dugas.ch/artificial_curiosity/GPT_architecture.html
- 7. 7 GPT-3 •Output sequence •Up to 2048 “guesses” at the most likely next word in the sequence ChatGPT Dr. Josh Locklair Source: https://dugas.ch/artificial_curiosity/GPT_architecture.html
- 8. 8 GPT-3 •Key advantages •Size of language model •Attention (technique for determining how important each token is to the output) ChatGPT Dr. Josh Locklair
- 9. 9 Risks of LLMs •LLMs absorb the worldview and biases of their human creators •LLMs “blindly” propagate information without consideration of human impact •LLMs cannot consider the truth of the text they generate ChatGPT Dr. Josh Locklair Source: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922
- 10. 10 Resources • https://dugas.ch/artificial_curiosity/GPT_architecture.html • An excellent high-level overview of the GPT-3 architecture • https://lambdalabs.com/blog/demystifying-gpt-3 • A slightly more technical summary of GPT-3 and related LLMs • http://jalammar.github.io/illustrated-gpt2/ • Attention and self-attention in the context of GPT-2 • https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 • Discussion of the risks associated with LLMs ChatGPT Dr. Josh Locklair