Open Source models security- Supply chain
- 1. 1 Supply Chain Security For Open Source Models
- 2. • M.sc in applied math • A year in Goethe universitat FFM • AI researcher in various domains • More than 10 patents AI & Cyber • Academic papers on AI & AI ethic • Official author at TDS • Representing Israel in QC stand. • A little of YouTube presence • Co-founder of LuminAI Security! natan.katz@gmail.com natan.katz@luminaisec.com https://www.linkedin.com/in/natan-katz-2936425/ J J
- 4. Before Hugging Face Until 20172018 models were nearly always a in-house product. • The organization aggregated data • Upon LOBʼs request data scientists trained models • Deployment- R&D with LOBʼs oversight Costs: • Data storage • DevOps • Cloud resources • Data scientists spent their time on “dark laborˮ Results depend on the amount of data and the quality of the work
- 5. Open Source Models • Attention is all you need 2017 • Model zoo began to appear Hugging Face What does the organization gain? • Data scientists need significantly less data • Flexible : Extremely easy to modify their task
- 6. AI Security Before ChatGPT The focus is mainly on the model files: • Pickle files are wrapped by operational code • This code can be vulnerable
- 7. SAAS Chat GPT November 2022 • OpenAI releases ChatGPT • Some other players appear Anthropic)
- 8. New Security Challenges • Chat GPT is used by all the organization • CISOs begin to ask questions • A new set of security challenges was born OWASP LLM 2023 InputOutput handling Data Leakage Prevention
- 11. Open Source Models - Benefits • Privacy - You don't share your data • R&D Efforts - It is extremely easy to customize • BI Flexibility - Easy to modify the model upon the organization needs • Cost - Only the cloud
- 14. Everyone can upload and even fake his affiliation Data scientists hardly ever care about security Security?! - “They lost to Bayern last week didn't they? I prefer basketballˮ
- 15. Open Source Models - Risks • Free upload- Everyone can upload and even fake his affiliation Meta • Web Security tests - Hardly exists • Hackersʼ motivationDevelopers rely on open-source. Attackers use that. • Data scientists don't care too much about security
- 16. What do CISOs think? • CISOs rarely talk to ds “I don't know about open source model. I guess there are noˮ • CISOs can take the hard decision: “Open-source models are risky we don't useˮ No AI. • CISOs focus mainly on the data itself and not on the models “We have rule based solutions for PI & PII“
- 17. Securing AI Development. Protecting AI-Powered Applications. OWASP ML
- 18. Poisoned Models