Securely Boosting Any Product with Generative AI APIs - Ruben Sitbon, Theodo Fintech
Download as PPTX, PDF0 likes29 views
The document, presented by Ruben Sitbon, outlines the evolution of generative AI (GenAI) and provides best practices for implementing successful GenAI products by defining dimensions and segmenting skill sets. It emphasizes the importance of customizing outputs, assessing security, and utilizing existing technologies to improve development processes. Steps are provided for various prompt levels and further resources for learning and development in this domain.
Securely Boosting Any Product with Generative AI APIs - Ruben Sitbon, Theodo Fintech
- 3. Who am I? @rubensitbon ruben-sitbon ● Started as Software Engineer ● Now Lead Solutions Architect Client examples: 3
- 4. GenAI in 2023 Means Chatbots Only a few game changing products 4
- 5. GenAI had failures in 2023 5
- 6. How to avoid bad GenAI products? Right Solution Right Team 6
- 7. Output customization Step 1 : Define the dimensions needed to solve the problem 7
- 8. Output customization Range of Knowledge Step 1 : Define the dimensions needed to solve the problem 8
- 9. Product Manager Skill set Software Engineer Skill set AI Engineer Skill set Research Realm Output customization Range of Knowledge Step 1 : Define the dimensions needed to solve the problem Step 2 : Segment with your team skill set 9
- 10. Product Manager Skill set Software Engineer Skill set AI Engineer Skill set Research Realm Output customization Range of Knowledge Step 1 : Define the dimensions needed to solve the problem Step 2 : Segment with your team skill set Step 3 : Position the tools on the chart ChatGPT GPTs RAG Functions Fine tuning Custom Models 10
- 11. Step 4 : Draw the area for each tool Product Manager Skill set Software Engineer Skill set AI Engineer Skill set Research Realm Output customization Range of Knowledge Step 1 : Define the dimensions needed to solve the problem Step 2 : Segment with your team skill set Step 3 : Position the tools on the chart ChatGPT GPTs RAG Functions Fine tuning Custom Models 11
- 12. Boost existing products instead of building new (bad) ones Product Ops GenAI API SwE 12
- 13. Help the technical team assess the security of the product ideas Facilitate the product design by supplying the right tools Develop scalable GenAI implementation 13
- 14. Help the technical team assess the security of the product ideas Facilitate the product design by supplying the right tools Develop scalable GenAI implementation 14
- 15. Facilitate the product design by supplying the right tools 15
- 16. Facilitate the product design by supplying the right tools 16
- 17. Facilitate the product design by supplying the right tools 17
- 18. Facilitate the product design by supplying the right tools 18
- 19. Facilitate the product design by supplying the right tools 19
- 20. Help the technical team assess the security of the product ideas Facilitate the product design by supplying the right tools Develop scalable GenAI implementation 20
- 21. Help the technical team assess the security of the product ideas Increase observability to monitor security issues Leverage on existing best practices Test a lot and limit the attack surface SQL/Prompt Injection DDos/EDos Attacks Supply Chain Vulnerability Custom Middlewares Monitor input Monitor output Manual testing Integration test API First architecture 21
- 22. Help the technical team assess the security of the product ideas Facilitate the product design by supplying the right tools Develop scalable GenAI implementation 22
- 23. Level 1: 0-Shot Prompt LLM Prompt Answer 23
- 24. Level 2: Few-Shot Prompt LLM Prompt Give context on how to answer Answer 24
- 25. Level 3: Simple Functions LLM Prompt Give context on how to answer Functions Develop/ Maintain Answer LLM 25
- 26. Level 4: Retrieval Augmented Generation LLMs Prompt Answer Vector DB Document Processing Upload Documents Upload Documents Store Chucked and vectorised documents Query DB 26
- 27. Level 5: RAG + Functions LLMs + Functions Prompt Answer Vector DB Document Processing Store Chucked and vectorised documents 27 Upload Documents Upload Documents Query DB
- 28. Let’s wrap it up 28 Mental model to find the right solution Improve each steps of the development process
- 29. What’s next! ● Try this library at home and show the demo to your PM ● LangChain Cookbook ● Youtube Channels like : Nordic APIs 29 theodo-fintech/nestjs-generative-ai
Editor's Notes
- #1: Hi Everyone Thanks for being here today and thanks for Nordic APIs to host this event So today we are going to talk about APIs of course and about AI since it’s the AI stage but first let me share something with you
- #2: Do you know what this means ? Maybe this would help ? I’ve heard some “ahhh” Shu ha ri are the 3 steps of learning to mastery of a skills
- #3: So why am I telling you all this ? Let’s introduce who I am I’m Ruben, I’m working at Sipios a french company which develop apps for Fintechs I started as a software engineer, now i’m Lead Solution Architect so my job is to find the right solution for a given problem of my prospective clients
- #4: First we need to understand the context of GenAI in 2023 We’ve seen a lot of chatbots And only a few game changing products
- #5: We also had the chance to witness the first failures of GenAI like this one Where AirCanada, had to refund a client because of a discount policy it’s chatbot imagined
- #7: In order to do that we need to understand the rules, understand the tools we have First let’s define the common axis on which we will make decisions when it comes to choose the right GenAI solution The first axis is the output customisation. Do you need only the raw output of a LLM which is a string ? or a formatted output ? or piece of code ready to be interpreted
- #8: Then you need to take a look at the range of knowledge you need to solve your problem Do you need “common” knowledge of a LLM or you need access to private documents or you need a LLM trained with very specific knowledge like medicine or law
- #9: Step 2 is to segment this complexe range of solution with teams skillset Product Manager skill set : knows how to use online solutions
- #10: Then you take your tools and put them on the workbench here on your mental model Chatgpt : very easy GPTs a bit harder Function is when you give your LLM the hability to call a function which can do an API call for example A RAG is a retrieval augmented generation : basically it’s a LLM with a database of documents Fine-tuning and finally building your own custom model
- #11: Finally decide de area of use for each tool For example : a simple GPT can be made by a product person, but if you want your GPT to be able to call APIs and return formatted outputs maybe you’ll need a little bit of help for a software engineer. Same for a RAG : a software engineer could build one. But if it’s a software systel with high criticity maybe you’ll eed a little bit of help from a AI Engineer
- #12: So why should we use these tools? to build another chatbot or to improve our existing products? I my opinion we should improve our existing products. And the only profile that can bring together The needs of the products The power of GenAI APis And the Ops : the fact to deploy those new features in production It’s us : the software engineers !!
- #13: So what does it mean for us. Well we need to work on 3 parts of our job : Contribute to the product design by providing the right tools Help the technical team asses the security of the product ideas Finally when it comes to actually build the solution implement it in a scalable way. And one solutions is the LangChain framework
- #14: I remember the day i’ve show my product manager a UI kit library like material UI. I saw a sparkle in his eyes. He told me “wow i didn’t knew we could do this, and this.” It’s the law of instrument : when you only have a hammer all problems look like a nail Well if the only genai applications you see are chatbots well you’ll keep building chatbots Here is an example of something you can do
- #21: Like for the API and web development the OWASP has decided to release the top 10 for LLMs
- #26: Vector DB like : Pinecone, Qdrant, Chroma
- #28: So let’s see ! Have we become master of GenAI ? Not yet but we are on a good path ! 1) We started from understanding the rules and mapping the GENAI environment the right way 2) We broke the rules of designing only chatbots and find a way to help product team, technical team in a scalable and secured way 3) For the Ri part ? Well, We need to stay curious and continue to make progress on the first two parts to create our own rules.