Intro to LLMs Basics and their Safety Consideration
0 likes252 views
The document provides an overview of large language models (LLMs), outlining their fundamental principles, training processes, and applications in various fields. It also highlights significant safety and ethical concerns related to LLMs, including issues of privacy, misinformation, bias, and security vulnerabilities like prompt injection. The discussion emphasizes the importance of careful consideration and responsible usage of LLM technologies.
Intro to LLMs Basics and their Safety Consideration
- 1. Introduction to Large Language Models Basics and Safety Considerations Hossein A. Rahmani PhD Student, UCL AI Center University College London (UCL) hossein.rahmani.22@ucl.ac.uk rahmanidashti.github.io
- 5. LLMs took over the world! but what are they, and how do they work? Adapted from: https://www.youtube.com/watch?v=9vM4p9NN0Ts
- 6. Are LLMs something new?
- 7. Are LLMs something new? LLMs have been around for a while! Any examples?
- 8. Are LLMs something new? LLMs have been around for a while! Any examples? Google Keyword Suggestion Keyboard Keyword Suggestion Gmail Smart Compose
- 9. Are LLMs something new? Source: https://levelup.gitconnected.com/the-brief-history-of-large-language-models-a-journey-from-eliza-to-gpt-4-and-google-bard-167c614af5af
- 10. Large Language Models (LLMs) are a subset of Deep learning. Artificial Intelligence Machine Learning Deep Learning Large Language Models making machine intelligent making machines that learn method of machine learning a new model of DL
- 11. Large, general-purpose language models can be pre-trained and the fine-tuned for specific purpose
- 12. Training a dog
- 13. Sit Come Down Stay Basic commands, sufficient for everyday life, a good canine citizen How do you train a dog
- 14. What if we need specialised service dog, such as police dog? basic-commands trained dog police dog guide dog Special trainings
- 15. Similar idea applies to large language models Transformer Architecture Pre-Training Data (Unlabeled) Pre-trained Model Post-training Data (Labeled) Fine-tuned Model Pre-training Fine-tuning PaLM, Google Medical data https://blog.google/technology/health/ai-llm-medpalm-research-thecheckup/ Med-PaLM
- 16. Language modeling imagine the following task: predict the next word in a sequence { A trained language model can ___ } What words come next?
- 17. Language modeling imagine the following task: predict the next word in a sequence { A trained language model can ___ } Can we frame this as a machine learning problem? How? What words come next?
- 18. Language modeling imagine the following task: predict the next word in a sequence { A trained language model can ___ } Can we frame this as a machine learning problem? How? What words come next? { A trained language model can ___ } LLM Word Probability speak 0.064 generate 0.075 politics 0.001 … … walk 0.002
- 19. AR Language Models ● Task: Predict the next word LLM Adapted from: https://www.youtube.com/watch?v=9vM4p9NN0Ts
- 20. AR Language Models ● Task: Predict the next word ● Steps: a. tokenize (she likely prefers ___) LLM She likely prefers 1 2 3 Adapted from: https://www.youtube.com/watch?v=9vM4p9NN0Ts
- 21. AR Language Models ● Task: Predict the next word ● Steps: a. tokenize b. forward LLM She likely prefers 1 2 3 Adapted from: https://www.youtube.com/watch?v=9vM4p9NN0Ts
- 22. AR Language Models ● Task: Predict the next word ● Steps: a. tokenize b. forward c. predict probability of next token LLM She likely prefers 1 2 3 Adapted from: https://www.youtube.com/watch?v=9vM4p9NN0Ts
- 23. AR Language Models ● Task: Predict the next word ● Steps: a. tokenize b. forward c. predict probability of next token d. sample LLM She likely prefers 1 2 3 5 Adapted from: https://www.youtube.com/watch?v=9vM4p9NN0Ts
- 24. AR Language Models ● Task: Predict the next word ● Steps: a. tokenize b. forward c. predict probability of next token d. sample e. detokenize LLM She likely prefers 1 2 3 dogs 5 Adapted from: https://www.youtube.com/watch?v=9vM4p9NN0Ts
- 25. AR Language Models ● Task: Predict the next word ● Steps: a. tokenize b. forward c. predict probability of next token d. sample e. detokenize LLM She likely prefers 1 2 3 dogs 5 inference only Adapted from: https://www.youtube.com/watch?v=9vM4p9NN0Ts
- 26. Large language models are trained to solve common language problems, like … Text classification Question answering Document summarisation Text generation
- 27. … then be tailored to solve specific problems in different fields, like … Retail Finance Entertainment Trained with a relatively small size of field datasets
- 28. What are the benefits of using LLMs? A single model can be used for different task The fine-tune process required minimum filed data The performance is continuously growing with more data and parameters
- 29. LLM development vs. Traditional development ● No ML experience needed ● No training examples ● No need to train a model ● Think about prompt design ● Yes ML expertise needed ● Yes training examples ● Yes need to train a model ● Yes, compute time + hardware ● Think about minimizing a loss function LLM Development (using pre-trained APIs) Traditional ML development Source: https://www.youtube.com/watch?v=zizonToFXDs
- 30. Safety and ethical consideration Privacy Concerns The use of LLMs in sensitive applications raises concerns about data privacy and the potential for abuse. Hallucination and Misinformation LLMs can be used to generate convincing yet false information, potentially leading to the spread of misinformation. Bias and Fairness LLMs can reflect and amplify societal biases present in their training data, leading to unfair or discriminatory outputs. Prompt Injection and Jailbreaks LLMs could be misused for tasks like creating targeted spam, phishing attacks, or generating harmful content.
- 31. Bias in LLMs responses Wang, Peiyi, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. "Large language models are not fair evaluators." arXiv preprint arXiv:2305.17926 (2023). "Simply changing the order of candidate responses leads to overturned comparison results, even though we add the command “ensuring that the order in which the responses were presented does not affect your judgment” into the prompt."
- 32. Hallucination in LLMs responses 1. Zhang, Yue, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang et al. "Siren’s song in the AI ocean: A survey on hallucination in large language models, 2023." URL https://arxiv. org/abs/2309.01219 (2024). 2. Wei, Jason, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. "Chain-of-thought prompting elicits reasoning in large language models." Advances in neural information processing systems 35 (2022): 24824-24837. "Three types of hallucinations occurred in LLM responses"
- 33. Hallucination in LLMs responses 1. Zhang, Yue, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang et al. "Siren’s song in the AI ocean: A survey on hallucination in large language models, 2023." URL https://arxiv. org/abs/2309.01219 (2024). 2. Wei, Jason, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. "Chain-of-thought prompting elicits reasoning in large language models." Advances in neural information processing systems 35 (2022): 24824-24837. "Three types of hallucinations occurred in LLM responses" "Chain-of-thought prompting enables LLMs to tackle complex arithmetic, commonsense, and symbolic reasoning tasks. Chain-of-thought reasoning processes are highlighted."
- 34. Privacy issues https://www.bitdefender.com/en-gb/blog/hotforsecurity/chatgpt-bug-leaks-users-chat-histories https://www.bbc.co.uk/news/technology-65047304 ChatGPT bug leaked users' chat histories "A ChatGPT glitch allowed some users to see the titles of other users' conversations, the artificial intelligence chatbot's boss has said." – On social media sites Reddit and Twitter, users had shared images of chat histories that they said were not theirs. How about personalized information in prompt or recommendation?
- 35. Prompt injection and jailbreak attack https://www.tigera.io/learn/guides/llm-security/prompt-injection/ https://outrider.org/nuclear-weapons/articles/could-chatbot-teach-you-how-build-dirty-bomb Prompt injection is a security risk where attackers manipulate the input prompts to an LLM to elicit undesirable or harmful responses.
- 36. Prompt injection and jailbreak attack Prompt injection is a security risk where attackers manipulate the input prompts to an LLM to elicit undesirable or harmful responses. https://www.tigera.io/learn/guides/llm-security/prompt-injection/ https://outrider.org/nuclear-weapons/articles/could-chatbot-teach-you-how-build-dirty-bomb
- 37. Prompt injection and jailbreak attack Prompt injection is a security risk where attackers manipulate the input prompts to an LLM to elicit undesirable or harmful responses. https://www.tigera.io/learn/guides/llm-security/prompt-injection/ https://outrider.org/nuclear-weapons/articles/could-chatbot-teach-you-how-build-dirty-bomb