Finetuning GenAI For Hacking and Defending
1 like330 views
The document outlines an agenda for a hands-on workshop focused on generative AI (GenAI) and security. Topics include understanding GenAI fundamentals, architectures of large language models (LLMs), and practical sessions on penetration testing and securing GenAI applications. The workshop aims to enhance knowledge in AI-related security measures and vulnerabilities.
![[BOS] (beginning of sequence): This token marks the start of a text. It
signifies to the LLM where a piece of content begins.
[EOS] (end of sequence): This token is positioned at the end of a text,
and is especially useful when concatenating multiple unrelated texts,
similar to <|endoftext|>. For instance, when combining two different
Wikipedia articles or books, the [EOS] token indicates where one article
ends and the next one begins.
[PAD] (padding): When training LLMs with batch sizes larger than one,](https://image.slidesharecdn.com/genaiworkshop-240717060935-474126c7/85/Finetuning-GenAI-For-Hacking-and-Defending-41-320.jpg)
Finetuning GenAI For Hacking and Defending
- 1. Jitendra Chauhan Join Whatsapp Group for coordination Welcome to GenAI Security Hands-on Workshop Co-Founder Detoxio AI 19 Years of R&D, AI/ML, Product Mgmt, 2x Patents, 2x Startups
- 2. Agenda Understand GenAI - History, Evolution and Fundamentals Demystify - AI, GenAI, and LLMs LLMs - Intuitive Understanding LLMs - Internal Architecture Run a Model (Hands On) Understand Key Parameters of LLMs Penetration Testing and Red Teaming LLMs GenAI Threat Model LLM Model Vulnerabilities GenAI Apps Vulnerabilities (Owasp Top 2) Red Teaming a Model - Manual and Automated (Hands On) Scanning GenAI Apps - Burp, Chakra, and others (Hands On)
- 3. Agenda Use GenAI to Enhance Security TBD TBD Securing GenAI Applications Guardrails LLMOps
- 4. Foundation of GenAI Learn LLMs Internals
- 5. AI?
- 6. AI?
- 7. AI? Learn floor plan by itself Sense - Seeing, .. Detect - Dirt / Clean Cleaning - Wash, Brooming Avoid - Obstacles Move - Across Layout Upskill - Not possible Interact - Terminal Fault - Manual Repair Learning - Not Possible When? - Manual Command AI Not AI
- 9. Ultimate Goal of AI Sophia Robots matching Humans
- 10. Two Major Advancements Generation of Content - Text, Audio,... Understanding of Meaning - Text, Audio,... The arrival of the transformer architecture in 2017, following the publication of the "Attention is All You Need" paper, revolutionised generative AI.
- 11. Transformers
- 13. GenAI & LLMs
- 17. Intuitive Understanding of AI and LLMs
- 18. Applications
- 19. Predictive Models Examples Neural Networks Deep Learning Decision Trees
- 21. Generative Models Examples: GAN, LLM - GPT2, BERT,
- 22. What comes next? To protect the network from unauthorized access, it is crucial to implement strong <Guess me> How did you come with your next word?
- 23. What comes next? To protect the network from unauthorized access, it is crucial to implement strong <Guess me> How did you come with your next word?
- 24. What comes next? How did you come with your next word? did you see them before? Think 5 other possible words? Can you continue and further add more words or even a sentence? LLMs are next word prediction program!!
- 25. Complete the story Once upon a time, in a forest, a speedy rabbit and a slow tortoise decided to have a race. Confident in his swift legs, the rabbit darted ahead but soon became complacent and decided to take a nap midway...... Complete the above story in your own words
- 26. Understand or Encoder Complete the story Once upon a time, in a forest, a speedy rabbit and a slow tortoise decided to have a race. Confident in his swift legs, the rabbit darted ahead but soon became complacent and decided to take a nap midway...... The diligent tortoise, though slow, continued steadily and eventually passed the sleeping rabbit, crossing the finish line first. The story teaches that consistent effort and perseverance can triumph over arrogance and laziness. Generate or Decoder LLMs Encode and Decode !!!
- 27. How AI Learns?
- 33. Temperature
- 41. [BOS] (beginning of sequence): This token marks the start of a text. It signifies to the LLM where a piece of content begins. [EOS] (end of sequence): This token is positioned at the end of a text, and is especially useful when concatenating multiple unrelated texts, similar to <|endoftext|>. For instance, when combining two different Wikipedia articles or books, the [EOS] token indicates where one article ends and the next one begins. [PAD] (padding): When training LLMs with batch sizes larger than one,
- 42. BPE Tokenizer
- 44. Self Attention
- 46. RNN - Encoder / Decoder
- 48. The “Self” in Self Attention Transformer Architecture - Self Attention
- 51. What to do when AI Fails?
- 53. Hugging Face Explore Open Source Models Run a Model on Kaggle Good Llama vs Bad Llama
- 54. 100K+ Models
- 55. Resonsible AI Overview RAG Architecture Pokebot - Poisioned GenAI App GenAI Security Testing
- 56. GenAI Apps Overview RAG Architecture Pokebot - Poisioned GenAI App GenAI Security Testing
- 57. LLM Challenges Key Challenges Large language models (LLMs) do not have access to the Updated and Latest Knowledge and Facts. LLMs can also face challenges with complex math problems and tend to generate text even when they don't know the answer (hallucination).
- 58. GenAI Apps The Retrieval Augmented Generation (RAG) framework overcomes these issues by connecting LLMs to external data sources and applications. Reseasoning using Chain of Thoughts Prompting the model to think more like a human by breaking down the problem into steps has shown success in improving reasoning performance. Chain of thought prompting involves including intermediate reasoning steps in examples used for oneor few-shot inference. ReAct : Reasoning and Action (Decision Making Process) ReAct combines chain of thought reasoning with action planning in LLMs. Examples include a question, thought (reasoning step), action (pre-defined set of actions), and observation (new information). Actions are limited to predefined options like search, lookup, and finish.
- 59. Pokebot - Sample RAG
- 63. Model Security GenAI App Security Data Security GenAI & LLM Security
- 64. LLM Security Model Vulnerabilities Build and Finetune Models LLM Red Teaming Securing LLMs LLM Data Poisioning
- 65. Case Study DBRX Red teaming
- 66. START Finetune Base LLM Design solution Build GenAI App No Yes Is Model Safe? No Yes Fix vulnerabilities Configure Monitoring & Guard Rails Red Team Guard Rails Is App Safe? No Yes Successful tests? Deploy on Production Prevent Data Leaks Red Team LLM Appsec Testing Secure LLM Secure App Monitor Prevent Secure GenAI Apps
- 67. GenAI In Security GenAI to assist SOC GenAI to assist Appsec (BurpGPT)
- 68. GenAI in SOC 1. Threat Detection and Response For XSOAR: Analysis of logs and network traffic to detect potential security threats. Automated generation of threat response scripts. 2. Security Policy Optimization: Creation of tailored security policies based on organizational requirements and threat landscape. Automated generation of security awareness training materials. 3. Code Generation with SAST Remediation: Automated generation of documentation and code from requirements or specifications. Generation of test cases and automation scripts with validation of false positives.
- 69. Tools And Technologies Vulnerability Management Tools: Nessus, OpenVAS Threat Intelligence Platforms: Splunk, AlienVault Security Orchestration Tools: Blue Team Field, Red Hat Automation Frameworks: Ansible, PowerShell,Chef Collaboration and Communication Tools: Slack, Jira
- 70. Pipeline Automation Threat Intelligence Collection Security Alerts Correlation Incident Response Initiation Vulnerability Scanning Threat Identification Threat Prioritization Automated Remediation Execution Vulnerability Patching Threat Mitigation System Recovery Planning [Business Continuity Management For IOC Automation RCA Analysis Matching with IOC and CVE Correlation of IOC For hosts Chef/Pupper for automated patch management Threat Mitigation with Mitigation and BCP Plan System Recovery Planning Business Continuity Management
- 71. Example TestCase Evaluate this test case and investigate it as soc analyst : powershell got executed with admin privileges at host 202.1.1.1, concerned active directory user was on vacation, et me know the detailed analysis and give me the chef or automation script to harden the windows machine which was executed in network,
- 72. GenAI in Appsec DAST pipeline can be automated with Burp Kinda tools. Example Pipeline would be through burp extension. Sample BURPGPT : Use the Azure OpenAI Service's API feature | BurpGPT Installation | BurpGPT
- 73. Sample Usecase Identifying potential vulnerabilities in web applications that use a crypto library affected by a specific CVE: Analyse the request and response data for potential security vulnerabilities related to the {CRYPTO_LIBRARY_NAME} crypto library affected by CVE-{CVE_NUMBER}: Web Application URL: {URL} Crypto Library Name: {CRYPTO_LIBRARY_NAME} CVE Number: CVE-{CVE_NUMBER} Request Headers: {REQUEST_HEADERS} Response Headers: {RESPONSE_HEADERS} Request Body: {REQUEST_BODY} Response Body: {RESPONSE_BODY} Identify any potential vulnerabilities related to the {CRYPTO_LIBRARY_NAME} crypto library affected by CVE-{CVE_NUMBER} in the request and response data and report them.
- 74. Sample Usecase -2 Scanning for vulnerabilities in web applications that use biometric authentication by analysing request and response data related to the authentication process: Analyse the request and response data for potential security vulnerabilities related to the biometric authentication process: Web Application URL: {URL} Biometric Authentication Request Headers: {REQUEST_HEADERS} Biometric Authentication Response Headers: {RESPONSE_HEADERS} Biometric Authentication Request Body: {REQUEST_BODY} Biometric Authentication Response Body: {RESPONSE_BODY} Identify any potential vulnerabilities related to the biometric authentication process in the request and response data and report them.
- 75. References
- 76. Name URL LLM Red Teaming of DBRX Shared Good Drive References LLM Red Teaming Notebook on Kaggle https://www.kaggle.com/code/jaycneo/llm -red-teaming-notebook-detoxio-ai Pokebot - Damn Vulnerable App https://huggingface.co/spaces/detoxioai/ Pokebot References
- 77. Hugging Face GPT https://huggingface.co/openai- community/gpt2 Attention What you Need https://arxiv.org/abs/1706.03762 Awesome LIst related to LLM and GenAI Security https://llmsecurity.net/ Learning GPT From Andrej Karapathi https://www.youtube.com/watch? v=zjkBMFhNj_g References
- 78. owasp_training_data_for_web.json · mahabharat/OWASP at main (huggingface.co) GitHub - aress31/burpgpt: A Burp Suite extension that integrates OpenAI's GPT to perform an additional passive scan for discovering highly bespoke vulnerabilities, and enables running traffic-based analysis of any type. https://chat.lmsys.org/ https://github.com/sindresorhus/awesome-chatgpt GitHub - Hannibal046/Awesome-LLM: Awesome-LLM: a curated list of Large Language Model References